-- Logs begin at Fri 2022-05-27 02:23:33 UTC, end at Fri 2022-05-27 03:42:24 UTC. -- May 27 03:09:08.821508 dv-cinder-828807-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 03:09:08.831462 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 03:09:08 2022] *** May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: nodename: dv-cinder-828807-4-iscsi May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: machine: x86_64 May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: clock source: unix May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: pcre jit disabled May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: detected number of CPU cores: 4 May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: current working directory: / May 27 03:09:08.833410 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: detected binary path: /usr/bin/uwsgi-core May 27 03:09:08.834067 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: your processes number limit is 47680 May 27 03:09:08.834067 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: your memory page size is 4096 bytes May 27 03:09:08.834067 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: detected max file descriptor number: 2048 May 27 03:09:08.834067 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: lock engine: pthread robust mutexes May 27 03:09:08.834067 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: thunder lock: enabled May 27 03:09:08.834067 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 03:09:08.834067 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 03:09:08.877778 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: Python main interpreter initialized at 0x55f2458b89c0 May 27 03:09:08.877778 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: python threads support enabled May 27 03:09:08.877778 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: your server socket listen backlog is limited to 100 connections May 27 03:09:08.877778 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: your mercy for graceful operations on workers is 90 seconds May 27 03:09:08.878099 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: mapped 403077 bytes (393 KB) for 2 cores May 27 03:09:08.878174 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: *** Operational MODE: preforking *** May 27 03:09:08.878446 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: *** uWSGI is running in multiple interpreter mode *** May 27 03:09:08.878446 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: spawned uWSGI master process (pid: 80263) May 27 03:09:08.878631 dv-cinder-828807-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 03:09:08.879677 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: spawned uWSGI worker 1 (pid: 80272, cores: 1) May 27 03:09:08.879677 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: spawned uWSGI worker 2 (pid: 80273, cores: 1) May 27 03:09:08.879677 dv-cinder-828807-4-iscsi devstack@keystone.service[80263]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 03:09:14.489474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:09:14.489474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: from cryptography.utils import int_from_bytes May 27 03:09:14.490280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:09:14.490280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: from cryptography.utils import int_from_bytes May 27 03:09:14.493604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:09:14.493604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: from cryptography.utils import int_from_bytes May 27 03:09:14.494072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:09:14.494072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: from cryptography.utils import int_from_bytes May 27 03:09:14.538929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:09:14.539118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:09:14.539312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] command line args: [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:09:14.539580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:09:14.539757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ================================================================================ {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:09:14.540032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] admin_token = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.540275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] config_dir = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.540461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.540644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] config_source = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.540823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] debug = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.540959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:09:14.541049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.541263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:09:14.541334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] command line args: [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:09:14.541397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.541548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.541641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:09:14.541767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ================================================================================ {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:09:14.541845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] admin_token = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] list_limit = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] config_dir = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] log_config_append = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] config_source = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] log_dir = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.542945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] debug = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] log_file = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.543907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] list_limit = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] log_config_append = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.544872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] log_dir = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] log_file = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.545991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.546837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] publish_errors = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.547986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] standard_threads = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.548915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] publish_errors = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] use_journal = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] use_json = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.549897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] use_stderr = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.550016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.550135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] use_syslog = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.550255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.550375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.550497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.550612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.550725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] standard_threads = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.550844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.550964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.551077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.551198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.551318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.551442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.551563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.551677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] use_journal = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.551677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] use_json = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.551933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.552056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] use_stderr = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.552056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] use_syslog = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.552274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.552389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:09:14.552513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.552670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.552786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.552900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.553889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.554945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.555998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.556960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.557916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.558836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.559974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.560896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.561947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.562947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.563928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.564972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565199 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.565903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.566944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.567890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.568960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.569997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.570289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.570289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.570511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.570659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.570776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.570898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.571973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.572907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.573986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.574102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.574216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.574331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.574445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] auth.external = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.575882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576337 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] auth.password = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] auth.token = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.576915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.577900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] auth.external = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] credential.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] auth.password = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.578964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] auth.token = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.579964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.580926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581048 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] credential.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581638 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.581984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.582898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.583985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.584975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.585901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.586962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.587933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588522 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.588989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.589913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.590027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.590775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.590903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.591970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.592084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.592211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.592211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.592428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.592800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.592917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.593977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.594917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.595990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.596414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.596538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.596655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.596772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.596887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.597960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.598909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.599899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600725 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.600957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601545 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.601899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.602957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.603916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.604983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.605920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606870 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.606987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607691 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.607976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608525 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.608987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] role.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.609939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] role.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.610999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.611972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] role.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] role.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.612908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.613970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.614907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.615890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.616944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.617991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.618938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619525 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.619886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.620950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.621885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.connection = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.622918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.623896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.connection = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.624854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.625396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.625512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.625635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.625753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.625868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.625983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.626913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80273) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:09:14.627454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80272) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:09:14.655778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.656034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.656258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.656610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.656849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.657079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.664795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.665738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.692674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.692903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.693553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.693790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.745925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.747218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.797853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.798091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.799226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.799462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.863806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.864994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.865121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.888863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.889960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.890041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.890172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.890254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.890388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.901485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.901636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.913785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.914963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.915092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.915172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80272) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.915673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80273) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:09:14.934647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.934844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:09:14.935483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.936209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.936383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:09:14.936549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:09:14.936655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.936768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.936867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:09:14.936980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.937083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.937167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.937231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.937376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.937490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.937490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.947405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.947684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.947771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.947980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.948079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.948266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.948348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.948588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.956890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.957289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.960176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.960455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.960563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.960731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.960877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.961008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.961135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.961285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.961411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.961538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.961672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.961876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.962976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.963072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.963273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.989417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.989767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.991136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.991421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.991561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.991736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.997246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.997482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.997646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:14.997766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.997925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:14.998207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.005033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.005320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.005413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.005685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.009778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.010153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.010235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.010451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.010544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.010737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.010818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.011988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.012678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.030846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.031075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.031158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.031352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.035788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.036020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.039416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.039630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.039715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.039954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.044830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.045114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.048433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.048721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.052209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.052501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.052584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.052791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.052872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.053075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.053158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.053358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.053440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.053641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.053734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.053920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.072938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.073221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.073302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.073488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.073572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.073750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.073832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.074078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.082293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.082596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.084031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.084316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.084397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.084598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.087337 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.087601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.087682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.087907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.087990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.088192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.088284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.088472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.088555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.088744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.102590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.102779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.102874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.103055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.103139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.103335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.103417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.103623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.103705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.103931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.104027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.104213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.104296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.104494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.122688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.123041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.125970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.126292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.129268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.129599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.130882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.131183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.132527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.132822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.132903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.133109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.133202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.133390 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.140503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.140779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.142098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.142366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.145363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.145646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.145728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.145930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.146010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.146208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.146292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.146477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.155804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.156032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.156130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.156319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.156484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.156705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.164328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80272) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.164610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.164693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80273) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:09:15.164892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.164973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.165171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.165252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.165443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.165526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.165709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.165790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.165986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.166082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.166254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.166336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.166525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.166657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.166791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.166872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80272) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.167947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.168126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80273) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:09:15.207112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f2458b89c0 pid: 80272 (default app) May 27 03:09:15.207319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f2458b89c0 pid: 80273 (default app) May 27 03:09:15.210431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f095ef-2a17-45d5-9316-40e58f4d5f31 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:15.210726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f095ef-2a17-45d5-9316-40e58f4d5f31 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:15.211005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f095ef-2a17-45d5-9316-40e58f4d5f31 None None] PATH_INFO: `/v3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:15.215957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Fri May 27 03:09:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:23.041532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a884-1639-46e6-a052-724580da0ed9 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.042067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a884-1639-46e6-a052-724580da0ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.042067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a884-1639-46e6-a052-724580da0ed9 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.047131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:23.052333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.052721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.052883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.119268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: INFO dbcounter [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] Registered counter for database keystone May 27 03:09:23.140014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG oslo_db.sqlalchemy.engines [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80272) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:09:23.146568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writer thread running {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:09:23.325700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG passlib.handlers.bcrypt [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80272) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:09:23.326042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG passlib.handlers.bcrypt [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80272) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:09:23.382019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:23.383650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:23.506068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 27 03:09:23.677802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1a884-1639-46e6-a052-724580da0ed9 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.746444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: INFO dbcounter [None req-b5e1a884-1639-46e6-a052-724580da0ed9 None None] Registered counter for database keystone May 27 03:09:23.765789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG oslo_db.sqlalchemy.engines [None req-b5e1a884-1639-46e6-a052-724580da0ed9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80273) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:09:23.772006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writer thread running {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:09:23.891891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1a884-1639-46e6-a052-724580da0ed9 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:23.896066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c91c3-a51b-487c-99d8-f1982a8335dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.896720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c91c3-a51b-487c-99d8-f1982a8335dc admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.897006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c91c3-a51b-487c-99d8-f1982a8335dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.897285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c91c3-a51b-487c-99d8-f1982a8335dc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.905761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c91c3-a51b-487c-99d8-f1982a8335dc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:23.956356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING keystone.common.rbac_enforcer.enforcer [None req-626c91c3-a51b-487c-99d8-f1982a8335dc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:09:23.956705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c91c3-a51b-487c-99d8-f1982a8335dc admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:23.967080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:09:23.973388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.988352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf1996-ae79-45d7-8578-5cc3839f997c None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:23.992714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22324de7-c489-4bb4-85b3-25a001274144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.993816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22324de7-c489-4bb4-85b3-25a001274144 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.994105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22324de7-c489-4bb4-85b3-25a001274144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.994384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22324de7-c489-4bb4-85b3-25a001274144 admin admin] PATH_INFO: `/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.996448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22324de7-c489-4bb4-85b3-25a001274144 admin admin] RBAC: Authorizing `identity:get_project(project_id=d52b88b861a746ccb689f0f6a32f3ad1)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:24.153901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING keystone.common.rbac_enforcer.enforcer [None req-22324de7-c489-4bb4-85b3-25a001274144 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:09:24.154547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22324de7-c489-4bb4-85b3-25a001274144 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:24.158933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:26.315080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d64937-4220-48c2-9d6f-14ceef987cc6 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.315564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d64937-4220-48c2-9d6f-14ceef987cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.315564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d64937-4220-48c2-9d6f-14ceef987cc6 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.319649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:26.324936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97a4c-4d10-4a3b-93d0-b8a076bb3e44 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.325353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97a4c-4d10-4a3b-93d0-b8a076bb3e44 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.325505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97a4c-4d10-4a3b-93d0-b8a076bb3e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.344897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4ac97a4c-4d10-4a3b-93d0-b8a076bb3e44 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:26.346403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4ac97a4c-4d10-4a3b-93d0-b8a076bb3e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:26.354136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 03:09:26.376465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d64937-4220-48c2-9d6f-14ceef987cc6 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.388104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d64937-4220-48c2-9d6f-14ceef987cc6 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.392220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf720-8121-4884-a1bb-db0e59ea3208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.392838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf720-8121-4884-a1bb-db0e59ea3208 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.393123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf720-8121-4884-a1bb-db0e59ea3208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.393398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf720-8121-4884-a1bb-db0e59ea3208 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.400946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bf720-8121-4884-a1bb-db0e59ea3208 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:26.403220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bf720-8121-4884-a1bb-db0e59ea3208 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:26.411647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:09:26.416814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97a4c-4d10-4a3b-93d0-b8a076bb3e44 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.426134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97a4c-4d10-4a3b-93d0-b8a076bb3e44 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.430475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c11763-cbfb-4288-9ee6-8ca890a73954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.431327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c11763-cbfb-4288-9ee6-8ca890a73954 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.431612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c11763-cbfb-4288-9ee6-8ca890a73954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.431928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c11763-cbfb-4288-9ee6-8ca890a73954 admin admin] PATH_INFO: `/v3/users/582245e4f2eb472dbbc5854ee9bc3452` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.433318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c11763-cbfb-4288-9ee6-8ca890a73954 admin admin] RBAC: Authorizing `identity:get_user(user_id=582245e4f2eb472dbbc5854ee9bc3452)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:26.435142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c11763-cbfb-4288-9ee6-8ca890a73954 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:26.436721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/users/582245e4f2eb472dbbc5854ee9bc3452 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:28.670059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbeb50-8928-4d0b-bc44-56e6b1a1d5ef None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.670600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbeb50-8928-4d0b-bc44-56e6b1a1d5ef None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.670695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbeb50-8928-4d0b-bc44-56e6b1a1d5ef None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.672110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:28.677267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19267f-afd2-49a4-b634-49100863e438 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.677697 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19267f-afd2-49a4-b634-49100863e438 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.677836 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19267f-afd2-49a4-b634-49100863e438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.688574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b507a38-85d7-4d83-8cf8-88ebe78f673b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.688864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b507a38-85d7-4d83-8cf8-88ebe78f673b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.689143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b507a38-85d7-4d83-8cf8-88ebe78f673b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.690455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:28.695543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.695669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0e19267f-afd2-49a4-b634-49100863e438 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:28.695868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.696186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.696837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0e19267f-afd2-49a4-b634-49100863e438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:28.736376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG passlib.handlers.bcrypt [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80273) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:09:28.736824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG passlib.handlers.bcrypt [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80273) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:09:28.775162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:28.783302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:28.784924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:28.799171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19267f-afd2-49a4-b634-49100863e438 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.822696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:28.884026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:28.902380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19267f-afd2-49a4-b634-49100863e438 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.906193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.941012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.941845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.942138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.942418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.955201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:28.966855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:28.968246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:28.969081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:28.974761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.058130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.076232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc84c27-6049-44a6-b703-26a6ff0090e8 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.109934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.110599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.110888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.111159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.111958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:29.112270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.112906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80861ee7-5745-48f9-aed7-2212990d1d74 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.113660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.115170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.125329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:29.130998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.150721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.151632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.152485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.152766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.220304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.222523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.226226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c64948-0a83-48cc-a5c0-de5cfdf8a3b8 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.263117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.264028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.264310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.264588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] PATH_INFO: `/v3/users/582245e4f2eb472dbbc5854ee9bc3452` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.265802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] RBAC: Authorizing `identity:get_user(user_id=582245e4f2eb472dbbc5854ee9bc3452)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.267270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.268915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 9/16] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/users/582245e4f2eb472dbbc5854ee9bc3452 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:29.274154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.288652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.322862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 8/17] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:09:29.328494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.331121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.371551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e41b05-6af0-4303-8a48-d7acd54e19d7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.404640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.405423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.405709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.405991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.407174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.408663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.410255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:29.415929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.427349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753796-8cef-458f-b142-08876e980be9 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.459696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.460587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.460942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.461227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.480551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.481939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.493856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:29.502010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:29.502857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:29.503661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:29.503661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:29.503661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:29.503661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:29.503661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:29.509146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.514103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5fd07-f9ee-43e8-9b2a-c1d0329e7894 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.548496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca424fb4-70de-4187-963c-077879045929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.549156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424fb4-70de-4187-963c-077879045929 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.549440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424fb4-70de-4187-963c-077879045929 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.549720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424fb4-70de-4187-963c-077879045929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.550569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca424fb4-70de-4187-963c-077879045929 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.domain.id=default, user.id=582245e4f2eb472dbbc5854ee9bc3452) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:29.550879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca424fb4-70de-4187-963c-077879045929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.552369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca424fb4-70de-4187-963c-077879045929 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.559500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/role_assignments?user.id=582245e4f2eb472dbbc5854ee9bc3452&scope.domain.id=default&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:29.572856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.620350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e1709-2cbc-4316-8c46-a74f8e365879 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.656974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e4b78-13af-41d4-99f2-bd8f93322961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.657663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e4b78-13af-41d4-99f2-bd8f93322961 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.657953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e4b78-13af-41d4-99f2-bd8f93322961 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.658230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e4b78-13af-41d4-99f2-bd8f93322961 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.658975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e4b78-13af-41d4-99f2-bd8f93322961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:29.659286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e4b78-13af-41d4-99f2-bd8f93322961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.660733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e4b78-13af-41d4-99f2-bd8f93322961 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.679248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:32.122596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730315-7087-4bc2-b181-3ed091b00c7b None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.123644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730315-7087-4bc2-b181-3ed091b00c7b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.124353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf730315-7087-4bc2-b181-3ed091b00c7b None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.125938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:32.131433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591cce-20d5-46f1-bcab-4a709af28794 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.131855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591cce-20d5-46f1-bcab-4a709af28794 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.132025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591cce-20d5-46f1-bcab-4a709af28794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.154233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-74591cce-20d5-46f1-bcab-4a709af28794 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:32.155646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-74591cce-20d5-46f1-bcab-4a709af28794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:32.246258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 11/23] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:32.283209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf730315-7087-4bc2-b181-3ed091b00c7b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.418078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98123d-f74c-47e7-9f72-2e6510c72dc0 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.418704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98123d-f74c-47e7-9f72-2e6510c72dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.419180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98123d-f74c-47e7-9f72-2e6510c72dc0 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.420762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 12/24] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:32.429063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938283-cda1-4d5e-be4c-5f6ba4e1006c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.429614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938283-cda1-4d5e-be4c-5f6ba4e1006c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.430112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938283-cda1-4d5e-be4c-5f6ba4e1006c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.456556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf730315-7087-4bc2-b181-3ed091b00c7b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.468888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-51938283-cda1-4d5e-be4c-5f6ba4e1006c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:32.470382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-51938283-cda1-4d5e-be4c-5f6ba4e1006c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:32.507360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.508102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.508573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.508998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.532418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:32.540602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:32.541706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:32.542714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:32.544898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 13/25] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:32.550289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.551452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.560233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.601116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 13/26] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:32.636588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938283-cda1-4d5e-be4c-5f6ba4e1006c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.723777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a683af4-f217-41a3-87c7-f82a66ab58d9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.786420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719c223-31c5-43cd-9090-dc749855da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.787028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c223-31c5-43cd-9090-dc749855da02 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.787313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c223-31c5-43cd-9090-dc749855da02 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.787591 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c223-31c5-43cd-9090-dc749855da02 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.788402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c223-31c5-43cd-9090-dc749855da02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:32.788710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c223-31c5-43cd-9090-dc749855da02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.790133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c223-31c5-43cd-9090-dc749855da02 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.814849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 14/27] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:32.820318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e7c97-0831-47fd-ac39-9b3f256a3060 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.820615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e7c97-0831-47fd-ac39-9b3f256a3060 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.820905 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e7c97-0831-47fd-ac39-9b3f256a3060 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.822206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 15/28] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:32.835297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e7c97-0831-47fd-ac39-9b3f256a3060 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.860301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938283-cda1-4d5e-be4c-5f6ba4e1006c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.896887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b89300-88a6-49e6-84b3-0ce17d79a0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.897859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b89300-88a6-49e6-84b3-0ce17d79a0da None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.898338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b89300-88a6-49e6-84b3-0ce17d79a0da None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.898839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b89300-88a6-49e6-84b3-0ce17d79a0da None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.899784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b89300-88a6-49e6-84b3-0ce17d79a0da None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.901379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b89300-88a6-49e6-84b3-0ce17d79a0da None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.921122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.961616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e7c97-0831-47fd-ac39-9b3f256a3060 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.996134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa58a47-64f9-4cfd-8ba2-018a27f856c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.997276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58a47-64f9-4cfd-8ba2-018a27f856c6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.997722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58a47-64f9-4cfd-8ba2-018a27f856c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.998154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58a47-64f9-4cfd-8ba2-018a27f856c6 None admin] PATH_INFO: `/v3/users/582245e4f2eb472dbbc5854ee9bc3452` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.999560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa58a47-64f9-4cfd-8ba2-018a27f856c6 None admin] RBAC: Authorizing `identity:get_user(user_id=582245e4f2eb472dbbc5854ee9bc3452)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:33.001238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa58a47-64f9-4cfd-8ba2-018a27f856c6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:33.002995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 16/29] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/users/582245e4f2eb472dbbc5854ee9bc3452 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:33.004355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e759c-fea3-48db-a9b9-c292a8cb70e6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.004784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e759c-fea3-48db-a9b9-c292a8cb70e6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.005274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e759c-fea3-48db-a9b9-c292a8cb70e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.023192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c06e759c-fea3-48db-a9b9-c292a8cb70e6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:33.025058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c06e759c-fea3-48db-a9b9-c292a8cb70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:33.049861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 14/30] 10.222.0.62 () {60 vars in 1131 bytes} [Fri May 27 03:09:32 2022] POST /identity/v3/roles => generated 219 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:09:33.051535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623a5c-04f7-4252-b8a4-403cb54537f4 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.052075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623a5c-04f7-4252-b8a4-403cb54537f4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.052615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623a5c-04f7-4252-b8a4-403cb54537f4 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.054373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 15/31] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:33.055353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e623a5c-04f7-4252-b8a4-403cb54537f4 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:33.098456 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 17/32] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:33.100053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fcb46-669d-4328-83b1-bc0a5b5c370c None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.107848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fcb46-669d-4328-83b1-bc0a5b5c370c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.108069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:33.108731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fcb46-669d-4328-83b1-bc0a5b5c370c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.140835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ca0fcb46-669d-4328-83b1-bc0a5b5c370c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:33.142697 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ca0fcb46-669d-4328-83b1-bc0a5b5c370c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:33.191217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e623a5c-04f7-4252-b8a4-403cb54537f4 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:33.266432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:33.267250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.267537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.267814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.273488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:33.276339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:33.280469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 16/33] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:33.283759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:33.289517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:33.305185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 18/34] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:33.305730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fcb46-669d-4328-83b1-bc0a5b5c370c None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:33.450976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fcb46-669d-4328-83b1-bc0a5b5c370c None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:33.451192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63559d-35fe-4041-99d2-006f2e5960a1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:33.454693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:33.496914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:33.497867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.498311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.498731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.499631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:33.501004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:33.501378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:33.502023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.502377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.502727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] PATH_INFO: `/v3/domains/default/users/582245e4f2eb472dbbc5854ee9bc3452/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.505608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=582245e4f2eb472dbbc5854ee9bc3452, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:33.507519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:33.518763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 17/35] 10.222.0.62 () {60 vars in 1131 bytes} [Fri May 27 03:09:33 2022] POST /identity/v3/roles => generated 225 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:33.525102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:33.630121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 19/36] 10.222.0.62 () {58 vars in 1361 bytes} [Fri May 27 03:09:33 2022] PUT /identity/v3/domains/default/users/582245e4f2eb472dbbc5854ee9bc3452/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:33.762078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d77b1-c43f-4b0d-8bd4-00811b9feb11 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:33.809575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0289e-06de-4a17-9874-51ec4aa45515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:33.810843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0289e-06de-4a17-9874-51ec4aa45515 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.811334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0289e-06de-4a17-9874-51ec4aa45515 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.811812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0289e-06de-4a17-9874-51ec4aa45515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.829404 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0289e-06de-4a17-9874-51ec4aa45515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:33.830914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0289e-06de-4a17-9874-51ec4aa45515 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-27c0289e-06de-4a17-9874-51ec4aa45515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:33.851255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:33.852162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:33.852976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:33.852976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:33.852976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:33.852976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:33.854998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 18/37] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:33.861870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:33.912931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112bd76-f025-4727-8042-d2691ce0b801 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.913432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112bd76-f025-4727-8042-d2691ce0b801 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.913863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112bd76-f025-4727-8042-d2691ce0b801 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.915863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 19/38] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:33.921597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd5358-848c-4f5f-adb9-43a5eccf230f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.922064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd5358-848c-4f5f-adb9-43a5eccf230f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.922595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd5358-848c-4f5f-adb9-43a5eccf230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.935542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:33.941446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-41dd5358-848c-4f5f-adb9-43a5eccf230f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:33.943129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-41dd5358-848c-4f5f-adb9-43a5eccf230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:33.960184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:34.002661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334fcfb-35c7-4117-b3ec-6d26c75c0774 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:34.017360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 20/39] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:34.019217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b85ab2-c9b0-4d3e-8248-f1b142f9e8fa None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:34.019505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b85ab2-c9b0-4d3e-8248-f1b142f9e8fa None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:34.019788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b85ab2-c9b0-4d3e-8248-f1b142f9e8fa None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:34.021301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 21/40] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:34.027298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fad5-bcfb-431b-b0dd-90d1b9f82801 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:34.027644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fad5-bcfb-431b-b0dd-90d1b9f82801 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:34.027892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fad5-bcfb-431b-b0dd-90d1b9f82801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:34.046773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3b86fad5-bcfb-431b-b0dd-90d1b9f82801 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:34.047992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3b86fad5-bcfb-431b-b0dd-90d1b9f82801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:34.057644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:34.059240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:34.059544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:34.060188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:34.060962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:34.061279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:34.062917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:34.088044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 20/41] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:34.088044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:34.127741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 22/42] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:34.128895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86fad5-bcfb-431b-b0dd-90d1b9f82801 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:34.191203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66138b6e-bf8e-4311-b5ce-8db44a78be8e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:34.206324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:34.219983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:34.231061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:34.231740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:34.232077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:34.232362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:34.233126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:34.234526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:34.245316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86fad5-bcfb-431b-b0dd-90d1b9f82801 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:34.250414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 21/43] 10.222.0.62 () {60 vars in 1131 bytes} [Fri May 27 03:09:34 2022] POST /identity/v3/roles => generated 223 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:34.251134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:34.315127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:34.316235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:34.316235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:34.316477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:34.317191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:34.319686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:34.361295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 23/44] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:09:34 2022] POST /identity/v3/projects => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:34.383369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fae989-5372-413e-8056-7b6ac0dc66f7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:34.427647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1bf5a-d377-4db9-8505-08abea88e15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:34.428533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1bf5a-d377-4db9-8505-08abea88e15e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:34.428823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1bf5a-d377-4db9-8505-08abea88e15e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:34.429103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1bf5a-d377-4db9-8505-08abea88e15e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:34.430591 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1bf5a-d377-4db9-8505-08abea88e15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:34.432162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1bf5a-d377-4db9-8505-08abea88e15e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:34.434228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 22/45] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:34.440008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:34.556781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676d35e-116d-4da4-9b1b-cc8517fded16 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:34.585286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:34.594007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b0084-3b86-4bc7-8c75-2e5b4b15be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:34.594818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0084-3b86-4bc7-8c75-2e5b4b15be5a None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:34.595746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0084-3b86-4bc7-8c75-2e5b4b15be5a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:34.595746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0084-3b86-4bc7-8c75-2e5b4b15be5a None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:34.596464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b0084-3b86-4bc7-8c75-2e5b4b15be5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:34.598143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b0084-3b86-4bc7-8c75-2e5b4b15be5a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:34.624987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 24/46] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:09:34 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:35.907728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668d8ab-a8b6-4aa6-8c5e-b68c604f1386 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:35.908281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668d8ab-a8b6-4aa6-8c5e-b68c604f1386 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:35.908363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668d8ab-a8b6-4aa6-8c5e-b68c604f1386 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:35.909713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 23/47] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:35.915103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193e4c2-3265-43aa-ae2b-165a46449b0e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:35.915490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193e4c2-3265-43aa-ae2b-165a46449b0e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:35.915706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193e4c2-3265-43aa-ae2b-165a46449b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:35.932074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9193e4c2-3265-43aa-ae2b-165a46449b0e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:35.933912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9193e4c2-3265-43aa-ae2b-165a46449b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:35.998999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 25/48] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:36.022315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668d8ab-a8b6-4aa6-8c5e-b68c604f1386 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.065907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.138915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668d8ab-a8b6-4aa6-8c5e-b68c604f1386 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.179473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.180227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.180522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.180807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.191355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:36.199103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:36.200094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:36.200881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 24/49] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:36.206113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193e4c2-3265-43aa-ae2b-165a46449b0e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.302667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193e4c2-3265-43aa-ae2b-165a46449b0e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.313952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.337819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.338494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.338786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.339102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.339888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:36.340207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.341612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.353056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 26/50] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:36.359198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.454805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477316e-4697-4ad3-82d3-f5795b5dc0be None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.456318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.490584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.491453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.491741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.492047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] PATH_INFO: `/v3/users/582245e4f2eb472dbbc5854ee9bc3452` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.493237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] RBAC: Authorizing `identity:get_user(user_id=582245e4f2eb472dbbc5854ee9bc3452)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.494699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.496895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 25/51] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/users/582245e4f2eb472dbbc5854ee9bc3452 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:36.502205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.599767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16757df0-1259-45d8-9000-b7bf1058081e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.635878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911e4db-9364-45b6-8848-f33de45d2394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.636692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911e4db-9364-45b6-8848-f33de45d2394 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.636980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911e4db-9364-45b6-8848-f33de45d2394 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.637259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911e4db-9364-45b6-8848-f33de45d2394 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.638421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911e4db-9364-45b6-8848-f33de45d2394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.639917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911e4db-9364-45b6-8848-f33de45d2394 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.641534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 27/52] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:36.647262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.750081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97a44d-0722-4b61-b960-c1113bce668f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.782197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7400cd-3964-462e-82e6-5e6da21b0073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.782845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7400cd-3964-462e-82e6-5e6da21b0073 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.783129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7400cd-3964-462e-82e6-5e6da21b0073 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.783406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7400cd-3964-462e-82e6-5e6da21b0073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.784310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7400cd-3964-462e-82e6-5e6da21b0073 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.domain.id=default, user.id=582245e4f2eb472dbbc5854ee9bc3452) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:36.784626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7400cd-3964-462e-82e6-5e6da21b0073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.786090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7400cd-3964-462e-82e6-5e6da21b0073 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.791753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 26/53] 10.222.0.62 () {56 vars in 1322 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/role_assignments?user.id=582245e4f2eb472dbbc5854ee9bc3452&scope.domain.id=default&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:36.797720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a085-4ecc-407e-9285-ea280c0efb3f None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.798120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a085-4ecc-407e-9285-ea280c0efb3f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.798476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a085-4ecc-407e-9285-ea280c0efb3f None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.799988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 28/54] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:36.805854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ddf0-6e63-47f8-90e8-e99b5351b158 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.806147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ddf0-6e63-47f8-90e8-e99b5351b158 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.806432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ddf0-6e63-47f8-90e8-e99b5351b158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.824453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fb10ddf0-6e63-47f8-90e8-e99b5351b158 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:36.826023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fb10ddf0-6e63-47f8-90e8-e99b5351b158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:36.855531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.897952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 27/55] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:36.922265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1a085-4ecc-407e-9285-ea280c0efb3f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.966857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:37.029183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1a085-4ecc-407e-9285-ea280c0efb3f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.060834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98b1a1-aada-41a1-a3f0-12974d61ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.061628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98b1a1-aada-41a1-a3f0-12974d61ef98 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.061916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98b1a1-aada-41a1-a3f0-12974d61ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.062198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98b1a1-aada-41a1-a3f0-12974d61ef98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.063371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98b1a1-aada-41a1-a3f0-12974d61ef98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.064862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98b1a1-aada-41a1-a3f0-12974d61ef98 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.066471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 29/56] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:37.072087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ddf0-6e63-47f8-90e8-e99b5351b158 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:37.167500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ddf0-6e63-47f8-90e8-e99b5351b158 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.200081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29806e7-c118-4015-8eb0-cd15438d35a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.200731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29806e7-c118-4015-8eb0-cd15438d35a8 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.201019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29806e7-c118-4015-8eb0-cd15438d35a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.201301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29806e7-c118-4015-8eb0-cd15438d35a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.202025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29806e7-c118-4015-8eb0-cd15438d35a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.203481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29806e7-c118-4015-8eb0-cd15438d35a8 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.229319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 28/57] 10.222.0.62 () {60 vars in 1140 bytes} [Fri May 27 03:09:37 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:39.383614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fc03f-aff4-483f-ab83-1735de69f2bb None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:39.384138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fc03f-aff4-483f-ab83-1735de69f2bb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:39.384243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fc03f-aff4-483f-ab83-1735de69f2bb None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:39.385574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 30/58] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:39.390650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b042-185d-40ec-aaf3-ed0fed962fb6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:39.391044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b042-185d-40ec-aaf3-ed0fed962fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:39.391226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b042-185d-40ec-aaf3-ed0fed962fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:39.407692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-77e7b042-185d-40ec-aaf3-ed0fed962fb6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:39.408908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-77e7b042-185d-40ec-aaf3-ed0fed962fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:39.419685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:39.474515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 29/59] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:39.498978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fc03f-aff4-483f-ab83-1735de69f2bb None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:39.597547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fc03f-aff4-483f-ab83-1735de69f2bb None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:39.606722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:39.630705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208f7bc-1e5b-424f-a38d-a84111b1ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:39.631487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f7bc-1e5b-424f-a38d-a84111b1ef35 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:39.631774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f7bc-1e5b-424f-a38d-a84111b1ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:39.632086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f7bc-1e5b-424f-a38d-a84111b1ef35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:39.633261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208f7bc-1e5b-424f-a38d-a84111b1ef35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:39.634753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208f7bc-1e5b-424f-a38d-a84111b1ef35 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:39.636379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 31/60] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:39.641799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7b042-185d-40ec-aaf3-ed0fed962fb6 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:39.736809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7b042-185d-40ec-aaf3-ed0fed962fb6 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:39.768201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ffff8-57a8-4506-8ff7-c88646513686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:39.768857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffff8-57a8-4506-8ff7-c88646513686 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:39.769142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffff8-57a8-4506-8ff7-c88646513686 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:39.769417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffff8-57a8-4506-8ff7-c88646513686 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:39.770123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ffff8-57a8-4506-8ff7-c88646513686 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:39.771574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ffff8-57a8-4506-8ff7-c88646513686 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:39.797057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 30/61] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:09:39 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:43.353206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d39d1-be27-45d7-8baa-5846a0a1f5d2 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.353206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d39d1-be27-45d7-8baa-5846a0a1f5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.353206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d39d1-be27-45d7-8baa-5846a0a1f5d2 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.354414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 32/62] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:43.358374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b62952-6817-49ec-b9ef-ed55ad642f81 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.358780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b62952-6817-49ec-b9ef-ed55ad642f81 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.358940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b62952-6817-49ec-b9ef-ed55ad642f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.369719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.384511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-26b62952-6817-49ec-b9ef-ed55ad642f81 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:43.386322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-26b62952-6817-49ec-b9ef-ed55ad642f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:43.435173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e66be-5c29-4023-8a84-fa653be76f94 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.435457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e66be-5c29-4023-8a84-fa653be76f94 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.435992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e66be-5c29-4023-8a84-fa653be76f94 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.437575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 33/63] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:43.450389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17907f8a-b822-4877-9210-0a5cfb7d0b64 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.451001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17907f8a-b822-4877-9210-0a5cfb7d0b64 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.451476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17907f8a-b822-4877-9210-0a5cfb7d0b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.469098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-17907f8a-b822-4877-9210-0a5cfb7d0b64 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:43.470904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-17907f8a-b822-4877-9210-0a5cfb7d0b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:43.472451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 31/64] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:43.473846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62686d-69e0-4cb8-b5e7-7e253eff3cbe None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.474143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62686d-69e0-4cb8-b5e7-7e253eff3cbe None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.474428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62686d-69e0-4cb8-b5e7-7e253eff3cbe None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.475690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 32/65] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:43.481204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af909f-776d-4d9b-b3e8-1338312ee1b6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.481494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af909f-776d-4d9b-b3e8-1338312ee1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.481777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af909f-776d-4d9b-b3e8-1338312ee1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.504861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-15af909f-776d-4d9b-b3e8-1338312ee1b6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:43.506045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-15af909f-776d-4d9b-b3e8-1338312ee1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:43.545765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 34/66] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:43.546611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17907f8a-b822-4877-9210-0a5cfb7d0b64 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.582635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 33/67] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:43.584114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad88c-6a41-4bcc-8633-b54a6576e544 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.584406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad88c-6a41-4bcc-8633-b54a6576e544 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.584684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad88c-6a41-4bcc-8633-b54a6576e544 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.586110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 34/68] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:43.587072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c68cd-d6a9-41de-80bd-a081aced127b None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.587424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c68cd-d6a9-41de-80bd-a081aced127b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.587707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c68cd-d6a9-41de-80bd-a081aced127b None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.592568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 35/69] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:43.592568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c68cd-d6a9-41de-80bd-a081aced127b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.654671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.657918 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17907f8a-b822-4877-9210-0a5cfb7d0b64 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.665110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.692820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3880bb0-0dcc-4abc-a917-1fff0ba2f12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.693622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3880bb0-0dcc-4abc-a917-1fff0ba2f12f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.693916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3880bb0-0dcc-4abc-a917-1fff0ba2f12f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.694200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c68cd-d6a9-41de-80bd-a081aced127b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.694300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3880bb0-0dcc-4abc-a917-1fff0ba2f12f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.705508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3880bb0-0dcc-4abc-a917-1fff0ba2f12f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-a3880bb0-0dcc-4abc-a917-1fff0ba2f12f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:43.714560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:43.715445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:43.716453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 35/70] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:43.717609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05331439-b33e-479a-be5c-ca1ef4ec96df None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.717609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05331439-b33e-479a-be5c-ca1ef4ec96df None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.717864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05331439-b33e-479a-be5c-ca1ef4ec96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.728750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b972e-569d-4c90-9cd1-be536058cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.730065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b972e-569d-4c90-9cd1-be536058cd66 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.730348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b972e-569d-4c90-9cd1-be536058cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.730627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b972e-569d-4c90-9cd1-be536058cd66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.734418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-05331439-b33e-479a-be5c-ca1ef4ec96df None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:43.735595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-05331439-b33e-479a-be5c-ca1ef4ec96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:43.740468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b972e-569d-4c90-9cd1-be536058cd66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-619b972e-569d-4c90-9cd1-be536058cd66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:43.748629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:43.749465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:43.750232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 36/71] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:43.751094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76dca8-a880-4c2c-8a70-62d25006ebbc None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.751377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76dca8-a880-4c2c-8a70-62d25006ebbc None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.751801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76dca8-a880-4c2c-8a70-62d25006ebbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.767633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-aa76dca8-a880-4c2c-8a70-62d25006ebbc None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:43.768819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-aa76dca8-a880-4c2c-8a70-62d25006ebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:43.804330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 36/72] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:43.805033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05331439-b33e-479a-be5c-ca1ef4ec96df None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.846266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 37/73] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:43.847183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76dca8-a880-4c2c-8a70-62d25006ebbc None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.899754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.906424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05331439-b33e-479a-be5c-ca1ef4ec96df None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.906622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.941872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28411398-0481-4efb-9114-2e465746964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.942668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411398-0481-4efb-9114-2e465746964c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.942954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411398-0481-4efb-9114-2e465746964c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.943239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411398-0481-4efb-9114-2e465746964c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.953300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28411398-0481-4efb-9114-2e465746964c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:43.955940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76dca8-a880-4c2c-8a70-62d25006ebbc None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-28411398-0481-4efb-9114-2e465746964c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:43.960920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:43.961772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:43.962539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 37/74] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:43.963161 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28411398-0481-4efb-9114-2e465746964c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.993757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.994421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.994706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.995548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.996343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:43.996665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.998074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.006552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 38/75] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.007351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.063563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28411398-0481-4efb-9114-2e465746964c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.096552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.097214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.097502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.097789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.098541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.098862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.100292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.102202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ba252-d9b9-4ee2-89e1-03549d96b5e3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.108747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 38/76] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.109993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.116342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.128831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.137074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.138058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.138349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.138636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.148513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:44.157149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:44.158222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:44.159025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 39/77] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:44.159643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.211530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfca64-8f99-4d69-832d-89b7039da290 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.244538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.245349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.245632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.245908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.247075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.248536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.250130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 39/78] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.250969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.259796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303999e5-b235-45aa-91f2-714fd15a984c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.292914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.293588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.293879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.294164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.294923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.295237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.296661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.304711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 40/79] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.305492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.333312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.338402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.350231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df422c6-ff35-48aa-88c5-6b543f2807eb None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.383257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.384157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.384508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.384799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.400180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.401651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.403209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e20311-125d-4d32-9c49-1f27e1623e5f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.403685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 40/80] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.404540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.439179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.440014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.440296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.440571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.441749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.443203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.444857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 41/81] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.445593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.505547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41c027-8a70-4ef6-90c9-56872fd2356f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.537301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.537965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.538254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.538535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.539293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.539606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.540992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52193be-9c36-4af7-b69f-d232b8017cb0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.541098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.544309 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.549399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 41/82] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.550192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.559170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.582263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.582938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.583231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.583514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.584276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.585769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.605412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 42/83] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:09:44 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:44.606414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.654567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638e8e-0046-4741-9939-a56dfe547cd5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.692905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.693769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.694093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.694403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] PATH_INFO: `/v3/users/582245e4f2eb472dbbc5854ee9bc3452` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.695904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] RBAC: Authorizing `identity:get_user(user_id=582245e4f2eb472dbbc5854ee9bc3452)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.697537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.699851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 42/84] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/users/582245e4f2eb472dbbc5854ee9bc3452 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.700276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.727853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0d72-4afd-49f0-bbd6-b60ba78fd42b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.764972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.765649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.765946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.766224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.783995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.801038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.819470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d3550-7c26-4b34-b088-64007a97221b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.855872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.855872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.861446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.862107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.862395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.862682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.951710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.953549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.990749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 43/85] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:44.991958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.039589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.067904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 43/86] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:45.069097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.075210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.136864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307091f-027e-480b-9d9a-dd4caa8a705e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.173324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.174502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.174965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.175446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.176874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.178530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.180407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 44/87] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:45.181376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.181727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7efb8-0a46-4053-8240-c0f731639319 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.230984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.231577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.232322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.232322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.276775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.296210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33b3a6-f584-4309-87da-ff04dbbcef01 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.304081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.305565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.325294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.334455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.336183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.336948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.337431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.339924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.342254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.344767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 45/88] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:45.345780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.428151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 44/89] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:45.429372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.463022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93a9d4-4e5f-483b-bf7f-6a02c1d3d8cd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.511905 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.512702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.512987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.513268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] PATH_INFO: `/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.515634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] RBAC: Authorizing `identity:get_project(project_id=26ad0fc892fa4a8c8532d6b6e89ba872)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.517136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.518870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 46/90] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:45.519736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.521193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.561674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a97fc1-f5ae-488f-9fbe-2f3596358869 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.599520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.600212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.600507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.600716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.600815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.650898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac80bf3-028e-461d-937f-a0a59c919b1c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.686569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.688089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.699977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.701312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.701822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.702542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.703994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=c3a7322e88e7474ab0bccfd41305fae2) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:45.704963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.706860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.714674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 47/91] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/role_assignments?user.id=c3a7322e88e7474ab0bccfd41305fae2&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:45.716053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.771411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.801081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 45/92] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:45.802341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.860703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be119-3ff4-4e7d-9fa9-3516d75dc7ff None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.899963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.910778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.912257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.913207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.913724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.915759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.918723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.920686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 48/93] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:45.921791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.923144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42adcfd-08bd-4b0a-b67c-4571fa6de2d1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.001422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.002453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.002917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.003431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.004493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=26ad0fc892fa4a8c8532d6b6e89ba872, user.id=c3a7322e88e7474ab0bccfd41305fae2) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:46.004989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:46.006634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:46.025660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 46/94] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/role_assignments?user.id=c3a7322e88e7474ab0bccfd41305fae2&scope.project.id=26ad0fc892fa4a8c8532d6b6e89ba872&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:46.028790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.098371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:46.130081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356734b5-63ce-455e-afc5-7c2e9285e362 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.174964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdfc99-bf97-445a-8580-c620a5351d9a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.214376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.215373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.215671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.215983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.220640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:46.220848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f79ca-44c7-4b23-bca9-f02b8a1c6055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.221824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f79ca-44c7-4b23-bca9-f02b8a1c6055 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.222305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f79ca-44c7-4b23-bca9-f02b8a1c6055 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.222403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:46.222863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f79ca-44c7-4b23-bca9-f02b8a1c6055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.223949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f79ca-44c7-4b23-bca9-f02b8a1c6055 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=582245e4f2eb472dbbc5854ee9bc3452) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:46.224258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 49/95] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:46.224796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f79ca-44c7-4b23-bca9-f02b8a1c6055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:46.226546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f79ca-44c7-4b23-bca9-f02b8a1c6055 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:46.230718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.233923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 47/96] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/role_assignments?user.id=582245e4f2eb472dbbc5854ee9bc3452&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:46.304257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:46.345517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b54f2-366f-4464-9a4a-fb78136ae59a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.382675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d675b0-10d5-41f4-9bce-a2a8935e718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.383346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d675b0-10d5-41f4-9bce-a2a8935e718f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.383637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d675b0-10d5-41f4-9bce-a2a8935e718f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.383954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d675b0-10d5-41f4-9bce-a2a8935e718f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.384894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d675b0-10d5-41f4-9bce-a2a8935e718f None admin] RBAC: Adding query filter params (role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=c3a7322e88e7474ab0bccfd41305fae2) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:46.385215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d675b0-10d5-41f4-9bce-a2a8935e718f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:46.386808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d675b0-10d5-41f4-9bce-a2a8935e718f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:46.393346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 50/97] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/role_assignments?user.id=c3a7322e88e7474ab0bccfd41305fae2&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:47.172405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1e4e-0c50-4101-b36c-dade24959109 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.173506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1e4e-0c50-4101-b36c-dade24959109 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.174017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1e4e-0c50-4101-b36c-dade24959109 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.175606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 48/98] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:47.181256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4aa56-a254-4adc-90b7-681f041bc97d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.182692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4aa56-a254-4adc-90b7-681f041bc97d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.183185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4aa56-a254-4adc-90b7-681f041bc97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.202625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-19d4aa56-a254-4adc-90b7-681f041bc97d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:47.204222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-19d4aa56-a254-4adc-90b7-681f041bc97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:47.301495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 51/99] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:47.332368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb1e4e-0c50-4101-b36c-dade24959109 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.461094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb1e4e-0c50-4101-b36c-dade24959109 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.462999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:47.498830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.499637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.499962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.500249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.510809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:47.518579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:47.519572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:47.520479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 49/100] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:47.525710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4aa56-a254-4adc-90b7-681f041bc97d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.630733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:47.694731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4aa56-a254-4adc-90b7-681f041bc97d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.741736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.742414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.742707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.742990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.743844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:47.744200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.746325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.759475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 52/101] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:47.764837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.901394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b6728-55f3-4a98-8ce6-33725b52c0ab None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.948396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f65e1-cb22-4ce8-9af2-4ab52bb93192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.949291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f65e1-cb22-4ce8-9af2-4ab52bb93192 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.949778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f65e1-cb22-4ce8-9af2-4ab52bb93192 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.950239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f65e1-cb22-4ce8-9af2-4ab52bb93192 None admin] PATH_INFO: `/v3/users/7126ee0f823d41fbafeea61dbd855e3c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.966621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f65e1-cb22-4ce8-9af2-4ab52bb93192 None admin] RBAC: Authorizing `identity:get_user(user_id=7126ee0f823d41fbafeea61dbd855e3c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.968422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f65e1-cb22-4ce8-9af2-4ab52bb93192 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.970150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 50/102] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/users/7126ee0f823d41fbafeea61dbd855e3c => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:47.975798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.987245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667090af-ec53-4285-9860-6ea0b2494e7e None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.987527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667090af-ec53-4285-9860-6ea0b2494e7e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.987813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667090af-ec53-4285-9860-6ea0b2494e7e None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.989311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 51/103] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:48.005421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18acd4-4abe-4f91-8b7a-168d878e75b5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.005679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18acd4-4abe-4f91-8b7a-168d878e75b5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.005918 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18acd4-4abe-4f91-8b7a-168d878e75b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.028041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-de18acd4-4abe-4f91-8b7a-168d878e75b5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:48.030401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-de18acd4-4abe-4f91-8b7a-168d878e75b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:48.056552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.161668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 52/104] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:48.170127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc965c-c736-497e-8922-124696eb65d2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.214720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.233251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18acd4-4abe-4f91-8b7a-168d878e75b5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.251846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.251846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.251846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.251846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.411198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.412732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.433081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18acd4-4abe-4f91-8b7a-168d878e75b5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.502641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.503885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.504381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.504862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.506313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.508174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.510886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 53/105] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:48.515984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.551973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.587172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 53/106] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:48.597755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.636041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.724263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59368-a2d2-4b01-aa1c-8fd08a9927eb None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.786379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae1556-aaf7-48c4-a03f-349c0b73d929 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.792515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.793380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.798340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.798812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.799695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.801326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.843769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 54/107] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:09:48 2022] POST /identity/v3/projects => generated 302 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:48.855031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3331f42-36a8-4701-ba54-1a3fb1a68036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.856245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3331f42-36a8-4701-ba54-1a3fb1a68036 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.856696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3331f42-36a8-4701-ba54-1a3fb1a68036 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.857149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3331f42-36a8-4701-ba54-1a3fb1a68036 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.860934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3331f42-36a8-4701-ba54-1a3fb1a68036 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.862611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3331f42-36a8-4701-ba54-1a3fb1a68036 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.864659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 54/108] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:48.871286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.988689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.010281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543c12-c1af-4c90-9224-86c4fdf33361 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.048376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8878c7a-0926-41a0-ab65-8b430c2a30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.049047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878c7a-0926-41a0-ab65-8b430c2a30d8 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.049341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878c7a-0926-41a0-ab65-8b430c2a30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.049629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878c7a-0926-41a0-ab65-8b430c2a30d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.050490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8878c7a-0926-41a0-ab65-8b430c2a30d8 None admin] RBAC: Adding query filter params (role.id=55a716d562db424fbfa1add5c62abd9a, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=7126ee0f823d41fbafeea61dbd855e3c) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:49.050812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8878c7a-0926-41a0-ab65-8b430c2a30d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.052335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8878c7a-0926-41a0-ab65-8b430c2a30d8 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.058979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 55/109] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/role_assignments?user.id=7126ee0f823d41fbafeea61dbd855e3c&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=55a716d562db424fbfa1add5c62abd9a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:49.076561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0a5d-4ae3-4a5d-8976-8b382c2918da None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.077211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0a5d-4ae3-4a5d-8976-8b382c2918da None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.077707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0a5d-4ae3-4a5d-8976-8b382c2918da None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.079304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 55/110] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:49.085150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9b80a-a3ef-4411-ab02-0409ec6f379d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.085776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9b80a-a3ef-4411-ab02-0409ec6f379d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.086284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9b80a-a3ef-4411-ab02-0409ec6f379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.105643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-dbb9b80a-a3ef-4411-ab02-0409ec6f379d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:49.107266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-dbb9b80a-a3ef-4411-ab02-0409ec6f379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:49.208661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 56/111] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:49.246509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b0a5d-4ae3-4a5d-8976-8b382c2918da None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.372491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.395175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b0a5d-4ae3-4a5d-8976-8b382c2918da None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.439777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1d287-705f-4607-a6d1-20f7ad6da15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.441042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d287-705f-4607-a6d1-20f7ad6da15d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.441594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d287-705f-4607-a6d1-20f7ad6da15d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.442070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d287-705f-4607-a6d1-20f7ad6da15d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.459775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1d287-705f-4607-a6d1-20f7ad6da15d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-cad1d287-705f-4607-a6d1-20f7ad6da15d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:49.468473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:49.469319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:49.470116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:49.472532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 56/112] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:49.478519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9b80a-a3ef-4411-ab02-0409ec6f379d None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.558191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.565803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c6330-75f0-4ae8-8f06-cb9e658c6455 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.566147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c6330-75f0-4ae8-8f06-cb9e658c6455 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.566445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c6330-75f0-4ae8-8f06-cb9e658c6455 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.568234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 57/113] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:49.597865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d82824-626b-4757-9339-b544e3025ec9 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.598412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d82824-626b-4757-9339-b544e3025ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.598857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d82824-626b-4757-9339-b544e3025ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.617567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9b80a-a3ef-4411-ab02-0409ec6f379d None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.636847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b2d82824-626b-4757-9339-b544e3025ec9 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:49.638291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b2d82824-626b-4757-9339-b544e3025ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:49.696305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.697501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.698239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.698753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.699986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:49.700602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.702969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.713352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 57/114] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:49.719804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.753288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 58/115] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:49.794668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d82824-626b-4757-9339-b544e3025ec9 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.926277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cab1e-0a16-4463-810c-3894cf65c384 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.956318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.991925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.997676 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d82824-626b-4757-9339-b544e3025ec9 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.005302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92450a74-1f5e-4e87-9a34-3a30a94f266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.006210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92450a74-1f5e-4e87-9a34-3a30a94f266c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.006495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92450a74-1f5e-4e87-9a34-3a30a94f266c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.006774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92450a74-1f5e-4e87-9a34-3a30a94f266c None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.013200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92450a74-1f5e-4e87-9a34-3a30a94f266c None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.014781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92450a74-1f5e-4e87-9a34-3a30a94f266c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.018609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 58/116] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:50.019864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387f6e6-4e1b-4ea8-ab3b-e03f621af058 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.020156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387f6e6-4e1b-4ea8-ab3b-e03f621af058 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.020441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387f6e6-4e1b-4ea8-ab3b-e03f621af058 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.022082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 59/117] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:50.031863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8f6f2-e4f4-4cd8-a3a1-37332e913c5c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.031863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8f6f2-e4f4-4cd8-a3a1-37332e913c5c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.031863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8f6f2-e4f4-4cd8-a3a1-37332e913c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.059696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-12b8f6f2-e4f4-4cd8-a3a1-37332e913c5c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:50.060949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-12b8f6f2-e4f4-4cd8-a3a1-37332e913c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:50.076917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.077731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.078022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.078307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.096270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:50.105981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.107067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:50.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 59/118] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:50.108948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.180673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 60/119] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:50.181967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8f6f2-e4f4-4cd8-a3a1-37332e913c5c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.247379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e4d1-145c-4cae-89be-b6c166ffb944 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.286006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acebd6-d472-4e32-862e-b3ec0ccf41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.287008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acebd6-d472-4e32-862e-b3ec0ccf41cd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.287510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acebd6-d472-4e32-862e-b3ec0ccf41cd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.288093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acebd6-d472-4e32-862e-b3ec0ccf41cd None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.289492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acebd6-d472-4e32-862e-b3ec0ccf41cd None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.291216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acebd6-d472-4e32-862e-b3ec0ccf41cd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.293677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 60/120] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.295104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf60ba-f31b-4ed0-b268-981147231784 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.295594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf60ba-f31b-4ed0-b268-981147231784 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.296110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf60ba-f31b-4ed0-b268-981147231784 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.296339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8f6f2-e4f4-4cd8-a3a1-37332e913c5c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.297999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 61/121] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:50.302808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf60ba-f31b-4ed0-b268-981147231784 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.313215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.326944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.342789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.343475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.343762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.344105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.345103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.345421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.347688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.359697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 61/122] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.361472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.423269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf60ba-f31b-4ed0-b268-981147231784 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.458519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93640-5a74-4c51-b5bd-77a62f1a2b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.459640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93640-5a74-4c51-b5bd-77a62f1a2b92 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.460147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93640-5a74-4c51-b5bd-77a62f1a2b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.460638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93640-5a74-4c51-b5bd-77a62f1a2b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.474440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93640-5a74-4c51-b5bd-77a62f1a2b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-79d93640-5a74-4c51-b5bd-77a62f1a2b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:50.482811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.483660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:50.484751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:50.486952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 62/123] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:50.488463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04f3ea-0986-458f-a3d9-aae964b6660e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.488946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04f3ea-0986-458f-a3d9-aae964b6660e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.489432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04f3ea-0986-458f-a3d9-aae964b6660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.490969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88593f9-3cce-4f78-961c-c8e043c8ad1d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.507576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2c04f3ea-0986-458f-a3d9-aae964b6660e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:50.509101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2c04f3ea-0986-458f-a3d9-aae964b6660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:50.533639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.534786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.535076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.535355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0/users/c3a7322e88e7474ab0bccfd41305fae2/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.537987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=70324050ec23474180b5bb70f04382d0, user_id=c3a7322e88e7474ab0bccfd41305fae2, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.540119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.559290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 62/124] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:09:50 2022] PUT /identity/v3/projects/70324050ec23474180b5bb70f04382d0/users/c3a7322e88e7474ab0bccfd41305fae2/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:50.560523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.589034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.607990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 63/125] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:50.609265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04f3ea-0986-458f-a3d9-aae964b6660e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.623179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.700821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b150a8-b8f5-4ad6-923c-d07f3912a0b1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.745577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.746824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.747340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.747849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.749601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.751305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.753801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 63/126] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.759487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.769828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04f3ea-0986-458f-a3d9-aae964b6660e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.865223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.865925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.866372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.866490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.867248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.867543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.869062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.876569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713e8cb-5d03-4ce0-9e96-24e0e369a204 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.880393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 64/127] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.884955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.908212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.919198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.920072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.920359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.920637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.939434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:50.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.952331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:50.953159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:50.956248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 64/128] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:50.957586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.960599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.009258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429a15-a3d3-4e1b-93b9-3de26b82eda3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.046298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.047308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.047554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.047861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] PATH_INFO: `/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.049182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] RBAC: Authorizing `identity:get_project(project_id=26ad0fc892fa4a8c8532d6b6e89ba872)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.050705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.052695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 65/129] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.053554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.094518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4a8f9-dcf5-480e-8b17-37caf4625a81 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.139695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.141364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.142060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.142482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] PATH_INFO: `/v3/users/582245e4f2eb472dbbc5854ee9bc3452` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.144713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=582245e4f2eb472dbbc5854ee9bc3452)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.146598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.149048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 65/130] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/users/582245e4f2eb472dbbc5854ee9bc3452 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.149949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.169293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee534d-f3bc-40ad-9db1-c5f2d1b6fdee None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.198009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.208973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.209842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.210237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.210767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.211715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.212783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:51.213304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.214926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.233536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 66/131] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.235386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.321155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e4ead-916e-4f53-990c-fce5d7ff3ac2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.380282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.381237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.381530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.381811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] PATH_INFO: `/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872/users/c3a7322e88e7474ab0bccfd41305fae2/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.385341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ad0fc892fa4a8c8532d6b6e89ba872, user_id=c3a7322e88e7474ab0bccfd41305fae2, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.389504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.409006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e81a6a-b50f-4b58-ab3c-dc96abc25e32 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.429400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 66/132] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:09:51 2022] PUT /identity/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872/users/c3a7322e88e7474ab0bccfd41305fae2/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:51.430257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.447532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.448447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.448741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.449024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.453360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.454891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.457889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 67/133] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.466492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.537747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.587217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d157853-24d5-46a0-9c62-97371f75fea7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.627304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.628290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.628630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.628902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0/users/582245e4f2eb472dbbc5854ee9bc3452/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.631252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] RBAC: Authorizing `identity:create_grant(project_id=70324050ec23474180b5bb70f04382d0, user_id=582245e4f2eb472dbbc5854ee9bc3452, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.632839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.655577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 68/134] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:09:51 2022] PUT /identity/v3/projects/70324050ec23474180b5bb70f04382d0/users/582245e4f2eb472dbbc5854ee9bc3452/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:51.668740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb4f7f-4a94-4ce4-81d3-2d1142b1e184 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.670500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.719626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0ece9-353b-41ac-9462-3314b7d9ffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.720614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0ece9-353b-41ac-9462-3314b7d9ffaf None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.720911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0ece9-353b-41ac-9462-3314b7d9ffaf None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.721194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0ece9-353b-41ac-9462-3314b7d9ffaf None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.722808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0ece9-353b-41ac-9462-3314b7d9ffaf None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.724530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0ece9-353b-41ac-9462-3314b7d9ffaf None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.726285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 67/135] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.732861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.912822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10cef3-c751-4539-9f1b-5160f5bfbd87 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.939855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.963941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.964830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.968204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.968499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.972645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.974142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.975905 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 69/136] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:51.981673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229adaf9-2b42-444a-895d-c2d267923d02 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.981822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.982113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229adaf9-2b42-444a-895d-c2d267923d02 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.982279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229adaf9-2b42-444a-895d-c2d267923d02 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.983787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 68/137] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:51.995845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4504-344d-45b5-9094-2eb6f45ed41d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.995845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4504-344d-45b5-9094-2eb6f45ed41d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.995845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4504-344d-45b5-9094-2eb6f45ed41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.036430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-87ed4504-344d-45b5-9094-2eb6f45ed41d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:52.037768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-87ed4504-344d-45b5-9094-2eb6f45ed41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:52.115948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b313a-9b56-420b-b78b-04baafaf56a2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.193345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.194592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.195128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.195635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0/users/c3a7322e88e7474ab0bccfd41305fae2/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.199292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] RBAC: Authorizing `identity:create_grant(project_id=70324050ec23474180b5bb70f04382d0, user_id=c3a7322e88e7474ab0bccfd41305fae2, role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:52.202055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.247674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 70/138] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:09:51 2022] PUT /identity/v3/projects/70324050ec23474180b5bb70f04382d0/users/c3a7322e88e7474ab0bccfd41305fae2/roles/cabb92ea03d84bd399329e57ac452f9c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:09:52.304148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 69/139] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:52.352180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.429758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:52.492089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0bf87-8103-4450-a569-76822e8a8cfe None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.492255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0bf87-8103-4450-a569-76822e8a8cfe None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.492484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0bf87-8103-4450-a569-76822e8a8cfe None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.494000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 70/140] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:52.509384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd35052-bfed-443d-bee1-9b36f7d4ad88 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.509732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd35052-bfed-443d-bee1-9b36f7d4ad88 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.510615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd35052-bfed-443d-bee1-9b36f7d4ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.533746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035359fe-bb5a-4d10-bbed-2ec31b406717 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.569148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3fd35052-bfed-443d-bee1-9b36f7d4ad88 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:52.569755 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3fd35052-bfed-443d-bee1-9b36f7d4ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:52.607928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.608747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.609032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.609310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.610671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:52.612170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.613808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 71/141] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:52.620677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.696964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 71/142] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:52.752730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd35052-bfed-443d-bee1-9b36f7d4ad88 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.796977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:52.799417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6252a1-6d06-463c-91f4-fb6d8da8f7ff None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.821811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:52.860604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.861285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.861570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.861854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.862590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:52.864071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.903739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 72/143] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:09:52 2022] POST /identity/v3/users => generated 283 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:52.923855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd35052-bfed-443d-bee1-9b36f7d4ad88 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.970178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.971001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.971287 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.971569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.983537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:52.991960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:52.993007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:52.993766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 72/144] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:52.999547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.125095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01c9b1-6f7d-4e4b-8ecf-2d77d8c1d0ee None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.172056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.172786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.173080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.173361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.174136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:53.174448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.175953 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.188203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 73/145] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:53.194170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.306396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:53.317707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef25d2-958e-4599-b89e-166951c39cbc None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.356128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.357378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.357896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.358394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] PATH_INFO: `/v3/users/7126ee0f823d41fbafeea61dbd855e3c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.359850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] RBAC: Authorizing `identity:get_user(user_id=7126ee0f823d41fbafeea61dbd855e3c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.361534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.363374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 73/146] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/users/7126ee0f823d41fbafeea61dbd855e3c => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:53.368927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.413156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:53.497637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14f552-890f-4cfe-84aa-fa35eb46ac92 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.536447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fda1c-28eb-4ec1-a372-b210dc4605d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.537328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fda1c-28eb-4ec1-a372-b210dc4605d2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.537614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fda1c-28eb-4ec1-a372-b210dc4605d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.537889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fda1c-28eb-4ec1-a372-b210dc4605d2 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.539121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fda1c-28eb-4ec1-a372-b210dc4605d2 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.540645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fda1c-28eb-4ec1-a372-b210dc4605d2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.543009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 74/147] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:53.548837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.674869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4a59d-5033-48a5-a8c6-dad2219c1b36 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.722377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a2aa-447c-4aa5-898a-22f9a9e7a04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.723293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a2aa-447c-4aa5-898a-22f9a9e7a04b None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.723586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a2aa-447c-4aa5-898a-22f9a9e7a04b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.723881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a2aa-447c-4aa5-898a-22f9a9e7a04b None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0/users/7126ee0f823d41fbafeea61dbd855e3c/roles/55a716d562db424fbfa1add5c62abd9a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.729218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a2aa-447c-4aa5-898a-22f9a9e7a04b None admin] RBAC: Authorizing `identity:create_grant(project_id=70324050ec23474180b5bb70f04382d0, user_id=7126ee0f823d41fbafeea61dbd855e3c, role_id=55a716d562db424fbfa1add5c62abd9a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.730702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a2aa-447c-4aa5-898a-22f9a9e7a04b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.752517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 74/148] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:09:53 2022] PUT /identity/v3/projects/70324050ec23474180b5bb70f04382d0/users/7126ee0f823d41fbafeea61dbd855e3c/roles/55a716d562db424fbfa1add5c62abd9a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:54.169097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f937b31-3978-49d5-bc98-f66c4b63cb06 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.169459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f937b31-3978-49d5-bc98-f66c4b63cb06 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.169680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f937b31-3978-49d5-bc98-f66c4b63cb06 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.171075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 75/149] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:54.176899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52748037-a63a-45b9-aa9f-4f1b8173a9c5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.177201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52748037-a63a-45b9-aa9f-4f1b8173a9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.177466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52748037-a63a-45b9-aa9f-4f1b8173a9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.205207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-52748037-a63a-45b9-aa9f-4f1b8173a9c5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:54.206500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-52748037-a63a-45b9-aa9f-4f1b8173a9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:54.221843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.281551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 75/150] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:54.322885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f937b31-3978-49d5-bc98-f66c4b63cb06 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.441135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.445454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f937b31-3978-49d5-bc98-f66c4b63cb06 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.487146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69477b-3a61-49e2-857f-16cf75812420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.487890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69477b-3a61-49e2-857f-16cf75812420 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.488233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69477b-3a61-49e2-857f-16cf75812420 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.488512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69477b-3a61-49e2-857f-16cf75812420 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.502137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69477b-3a61-49e2-857f-16cf75812420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6f69477b-3a61-49e2-857f-16cf75812420 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:54.510019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:54.510877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:54.511645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 76/151] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:54.518532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52748037-a63a-45b9-aa9f-4f1b8173a9c5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.625425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52748037-a63a-45b9-aa9f-4f1b8173a9c5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.629231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec437-9ff3-44c3-88b7-d02a2965a53e None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.629520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec437-9ff3-44c3-88b7-d02a2965a53e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.629801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec437-9ff3-44c3-88b7-d02a2965a53e None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.631245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 77/152] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:54.644311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622207d-d3cd-4467-b45f-99da60e9d816 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.644594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622207d-d3cd-4467-b45f-99da60e9d816 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.644866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622207d-d3cd-4467-b45f-99da60e9d816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.672026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2622207d-d3cd-4467-b45f-99da60e9d816 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:54.673222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2622207d-d3cd-4467-b45f-99da60e9d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:54.702836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.703521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.703804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.704110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.705011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:54.705323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.711134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.727698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 76/153] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:54.736868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.760786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.810082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 78/154] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:54.841084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622207d-d3cd-4467-b45f-99da60e9d816 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.925612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc0e85-0687-4fdd-aebe-539599158ba0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.000211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.005157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.008086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.008086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.008086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.012696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.013106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622207d-d3cd-4467-b45f-99da60e9d816 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.014143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.016053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 77/155] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.025133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.077156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.077975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.078259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.078535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.101063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:55.121013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:55.122320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:55.123324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 79/156] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:55.127859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.156215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.186895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff320bd-76fc-410d-b103-dd585191f43d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.241903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.242571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.242862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.243143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.292926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80903bd9-fb50-44ec-ad73-71a667578970 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.339491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.345146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.356546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.357238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.357524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.357805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.358585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.358895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.360323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.376446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 80/157] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.377408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.394996 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.456269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.493065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 78/158] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:55.499104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.503694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c493e1c-22c9-41c9-8fc5-c1e82414754e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.539901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.540795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.541132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.541413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] PATH_INFO: `/v3/users/582245e4f2eb472dbbc5854ee9bc3452` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.548789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] RBAC: Authorizing `identity:get_user(user_id=582245e4f2eb472dbbc5854ee9bc3452)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.550223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.552266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 81/159] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/users/582245e4f2eb472dbbc5854ee9bc3452 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.560796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.642979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3096de-a063-4414-b7ed-72505cc859d1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.656175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.676709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaca1b-b82c-4de1-8a48-2e1d73d43fef None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.705948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.706838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.707129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.707410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.712970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.715164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.717493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 79/160] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:55.725190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.736792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.737467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.737754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.738033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.796141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.839218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.840749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.873593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff579883-9044-4e04-8272-598fb476da87 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.952523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dc43e-11cb-43c5-8828-c6ac6cbc7a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.953201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dc43e-11cb-43c5-8828-c6ac6cbc7a50 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.953490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dc43e-11cb-43c5-8828-c6ac6cbc7a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.953772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dc43e-11cb-43c5-8828-c6ac6cbc7a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.954629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dc43e-11cb-43c5-8828-c6ac6cbc7a50 None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=c3a7322e88e7474ab0bccfd41305fae2) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.954954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dc43e-11cb-43c5-8828-c6ac6cbc7a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.956442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dc43e-11cb-43c5-8828-c6ac6cbc7a50 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.963490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 80/161] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/role_assignments?user.id=c3a7322e88e7474ab0bccfd41305fae2&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.964770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5772c-33fd-448d-8ffa-c6b0d8415d96 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.965071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5772c-33fd-448d-8ffa-c6b0d8415d96 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.965352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5772c-33fd-448d-8ffa-c6b0d8415d96 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.966867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 81/162] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:55.968463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.977726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4749e-4463-4683-b302-1c37b716c3a2 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.978039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4749e-4463-4683-b302-1c37b716c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.978303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4749e-4463-4683-b302-1c37b716c3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.988486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 82/163] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:55.993900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.016863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fcf4749e-4463-4683-b302-1c37b716c3a2 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:56.018086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fcf4749e-4463-4683-b302-1c37b716c3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:56.158088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400e7f4-613b-494d-9446-b930e2d404d6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.175505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 82/164] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:56.176987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdc47b-6a76-4e3c-aecd-26317852bf1c None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.177282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdc47b-6a76-4e3c-aecd-26317852bf1c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.177565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdc47b-6a76-4e3c-aecd-26317852bf1c None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.179022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 83/165] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:56.197938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10082-4df3-4587-a0e2-5cf1da6ec274 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.198239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10082-4df3-4587-a0e2-5cf1da6ec274 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.198506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10082-4df3-4587-a0e2-5cf1da6ec274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.221254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.222139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.222420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.222696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.224617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.226074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.228053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 83/166] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:56.232111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9de10082-4df3-4587-a0e2-5cf1da6ec274 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:56.233305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9de10082-4df3-4587-a0e2-5cf1da6ec274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:56.233534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.304247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.333619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.365175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 84/167] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:56.366431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10082-4df3-4587-a0e2-5cf1da6ec274 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.439941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa161-caf9-491f-9c5d-749b297cbc96 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.494771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10082-4df3-4587-a0e2-5cf1da6ec274 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.508674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.509346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.509690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.510053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.510911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=582245e4f2eb472dbbc5854ee9bc3452) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:56.511225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.512825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.521601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 84/168] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/role_assignments?user.id=582245e4f2eb472dbbc5854ee9bc3452&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:56.525440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.542629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.543673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.544252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.544668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.569063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:56.578683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:56.579865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:56.580810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:56.585739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 85/169] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:56.592862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.645938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.724447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2bd2b-6fe4-475c-b93f-1993a64efafb None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.755148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.765746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86c0e-76f9-488e-9c39-8dc85eadbf35 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.780642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.781487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.781774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.782151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.797340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:56.814303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:56.815328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:09:56.816319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 85/170] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:56.823136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.823831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.824127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.824408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.825178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:56.825493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.826894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.828886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.840205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 86/171] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:56.849482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.980009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ddb0a-9215-45cb-91c9-733dd9d33fb5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.011625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.024878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b268-72d3-4517-9dc1-597a861ef620 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.040747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.042111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.042402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.042685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.043460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:57.043769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.045417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.054633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 86/172] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:57.060810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.062256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.063148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.063436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.063718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.064962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.066421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.068276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 87/173] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:57.084392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.124254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.211581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b536f-03c8-415b-9b70-7cf27d319261 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.230181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722b0be-35c1-4a97-9f65-761411a3d3ab None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.250746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.251415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.251701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.252015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.294472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.312550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.313459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.313746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.314022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] PATH_INFO: `/v3/users/c3a7322e88e7474ab0bccfd41305fae2` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.316625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] RBAC: Authorizing `identity:get_user(user_id=c3a7322e88e7474ab0bccfd41305fae2)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.318080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.319929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 87/174] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/users/c3a7322e88e7474ab0bccfd41305fae2 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:57.326499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.356885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.358363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.475896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7badb-81fb-4a14-82b8-1c025ded7326 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.489392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.520419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 88/175] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:09:57.526493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.573131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.573818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.574082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.574360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.640207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.698826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dededa-f3cc-4e8e-a01e-39ba260061fa None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.714823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.714823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.784471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.785687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.786207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.786684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] PATH_INFO: `/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.789105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] RBAC: Authorizing `identity:get_project(project_id=26ad0fc892fa4a8c8532d6b6e89ba872)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.790865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.792816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 89/176] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/projects/26ad0fc892fa4a8c8532d6b6e89ba872 => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:57.801111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.915051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 88/177] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:09:57.915696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.921899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.001043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a26c7-e288-4a1b-825a-962caa4674c2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.048967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.063440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.064140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.064430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.064712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.065567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=26ad0fc892fa4a8c8532d6b6e89ba872, user.id=c3a7322e88e7474ab0bccfd41305fae2) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.065886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.067343 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.082120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 90/178] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/role_assignments?user.id=c3a7322e88e7474ab0bccfd41305fae2&scope.project.id=26ad0fc892fa4a8c8532d6b6e89ba872&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:58.161873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfdeb6-2ae4-44b0-afb7-fd96a4999330 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.203045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8148c43-aa91-45dc-a0d6-e01c1e6f0d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.204393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8148c43-aa91-45dc-a0d6-e01c1e6f0d03 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.204929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8148c43-aa91-45dc-a0d6-e01c1e6f0d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.205413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8148c43-aa91-45dc-a0d6-e01c1e6f0d03 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.206831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8148c43-aa91-45dc-a0d6-e01c1e6f0d03 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.208561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8148c43-aa91-45dc-a0d6-e01c1e6f0d03 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.211798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 89/179] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.218771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.234600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccf84c-0c02-49cf-9240-4dbdcd839e95 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.235219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccf84c-0c02-49cf-9240-4dbdcd839e95 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.235728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccf84c-0c02-49cf-9240-4dbdcd839e95 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.237451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 90/180] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:58.249584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7598c-7480-461d-afe2-17fa630a5694 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.250202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7598c-7480-461d-afe2-17fa630a5694 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.250680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7598c-7480-461d-afe2-17fa630a5694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.275234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2fd7598c-7480-461d-afe2-17fa630a5694 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:58.279715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2fd7598c-7480-461d-afe2-17fa630a5694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:58.396201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.406651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86d325-7a63-4d63-bb28-06901cc7207c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.410031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 91/181] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:58.411418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d485ee-5828-43f6-bc33-2a9d602235cb None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.411727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d485ee-5828-43f6-bc33-2a9d602235cb None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.412011 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d485ee-5828-43f6-bc33-2a9d602235cb None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.420361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 92/182] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:58.438959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63adb-ec53-4c7e-9404-0e246397b8ec None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.439256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63adb-ec53-4c7e-9404-0e246397b8ec None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.439522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63adb-ec53-4c7e-9404-0e246397b8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.460997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-eca63adb-ec53-4c7e-9404-0e246397b8ec None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:58.462459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-eca63adb-ec53-4c7e-9404-0e246397b8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:58.473167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801c2d-5f3c-4a73-8177-9cfdec8068eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.473827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801c2d-5f3c-4a73-8177-9cfdec8068eb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.474105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801c2d-5f3c-4a73-8177-9cfdec8068eb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.474763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801c2d-5f3c-4a73-8177-9cfdec8068eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.475619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10801c2d-5f3c-4a73-8177-9cfdec8068eb None admin] RBAC: Adding query filter params (role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=c3a7322e88e7474ab0bccfd41305fae2) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.476255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10801c2d-5f3c-4a73-8177-9cfdec8068eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.478006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10801c2d-5f3c-4a73-8177-9cfdec8068eb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.488014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 91/183] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/role_assignments?user.id=c3a7322e88e7474ab0bccfd41305fae2&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.489600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf459da-8090-4d63-b5ae-143d94738b7a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.490229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf459da-8090-4d63-b5ae-143d94738b7a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.490518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf459da-8090-4d63-b5ae-143d94738b7a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.492435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 92/184] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:58.493367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf459da-8090-4d63-b5ae-143d94738b7a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.553807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 93/185] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:58.554929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39820cd-1f5b-4527-8a7a-9ebd1c7c0ba3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.555226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39820cd-1f5b-4527-8a7a-9ebd1c7c0ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.555508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39820cd-1f5b-4527-8a7a-9ebd1c7c0ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.574163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d39820cd-1f5b-4527-8a7a-9ebd1c7c0ba3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:58.575367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d39820cd-1f5b-4527-8a7a-9ebd1c7c0ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:58.575910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.637169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf459da-8090-4d63-b5ae-143d94738b7a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.661765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 94/186] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:58.663135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feed129-cd21-40be-96a7-f8196980eabd None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.663425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feed129-cd21-40be-96a7-f8196980eabd None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.663709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feed129-cd21-40be-96a7-f8196980eabd None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.665210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 95/187] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:58.670638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feed129-cd21-40be-96a7-f8196980eabd None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.684275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c2de-09e7-4e18-8926-1aface08ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.685516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c2de-09e7-4e18-8926-1aface08ec14 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.685804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c2de-09e7-4e18-8926-1aface08ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.686078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c2de-09e7-4e18-8926-1aface08ec14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.697850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c2de-09e7-4e18-8926-1aface08ec14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-7317c2de-09e7-4e18-8926-1aface08ec14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:58.705639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:58.706532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:58.707330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 93/188] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:58.711914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a1b16-222b-41a3-bb89-ae862c753bd5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.711914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a1b16-222b-41a3-bb89-ae862c753bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.711914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a1b16-222b-41a3-bb89-ae862c753bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.727222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fb8a1b16-222b-41a3-bb89-ae862c753bd5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:58.729042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fb8a1b16-222b-41a3-bb89-ae862c753bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:58.749723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.795469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feed129-cd21-40be-96a7-f8196980eabd None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.799195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 94/189] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:58.800705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a1b16-222b-41a3-bb89-ae862c753bd5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.831774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.832597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.832886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.833233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.834414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.835916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.837552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 96/190] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.838328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.847183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.899807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a1b16-222b-41a3-bb89-ae862c753bd5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.937275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.938078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.938363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.938481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae2e60-25f1-4ba2-90ef-1371f162388a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.938641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.948719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:58.956411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:58.957418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:58.958296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 95/191] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:58.958994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.966195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.975427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.976133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.976416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.976692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.977450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.977757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.979676 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.992178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 97/192] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.992178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.057157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5ce5a-d8c3-47fd-98b6-b0cd459ae8e5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.064195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.088711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde007b4-5151-4251-80a7-75606eb5bd77 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.091813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.092643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.092928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.093207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.103159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:59.111868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:59.112820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:09:59.113626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 96/193] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:59.114270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.126576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.127244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.127528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.127802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.128538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.129976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.151743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 98/194] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:09:58 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:59.152575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.204836 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.240495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685ede9-2ec1-447e-8fa8-c4ae3bf989a2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.273378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833f92d-6cd4-417f-ad7f-e52bc864acb9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.278396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.279088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.279405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.279682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.280482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:59.280795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.282230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.292229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 97/195] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.293341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.319234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.320651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.321645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] PATH_INFO: `/v3/users/7126ee0f823d41fbafeea61dbd855e3c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.323177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] RBAC: Authorizing `identity:get_user(user_id=7126ee0f823d41fbafeea61dbd855e3c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.324351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.326154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.331864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 99/196] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/users/7126ee0f823d41fbafeea61dbd855e3c => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.331864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.417121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4c2be-2267-4e88-a913-a57c0631e025 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.459042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41eb22-c9f3-4f2d-a62d-99b1ed2165d6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.460564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.461430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.461882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.462306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.463901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:59.464364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.465907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.475219 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 98/197] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.475546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.476451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.499415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.500498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.500890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.501290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] PATH_INFO: `/v3/users/0a28b4be4d684511a1e8a9dcf4511f94` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.517202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] RBAC: Authorizing `identity:get_user(user_id=0a28b4be4d684511a1e8a9dcf4511f94)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.518833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.520918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 100/198] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/users/0a28b4be4d684511a1e8a9dcf4511f94 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.521606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.578331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.590104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f0b6c-08c5-4445-93eb-30297e193ded None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.629243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.630055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.630570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.630951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.655356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305b05b-a11c-4299-8436-3bfc77b8a888 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.704042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.704924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.705210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.705491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] PATH_INFO: `/v3/users/0a28b4be4d684511a1e8a9dcf4511f94` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.707074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] RBAC: Authorizing `identity:get_user(user_id=0a28b4be4d684511a1e8a9dcf4511f94)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.709536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.709872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.711143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 101/199] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/users/0a28b4be4d684511a1e8a9dcf4511f94 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.711528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.711932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.763299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.828608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.839575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f2695-8bff-4e9d-969f-52f7ec4bd635 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.843166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 99/200] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:59.844039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.882379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.883039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.883427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.883712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.966166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.968559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.969973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb01ca8-ec4e-47b7-8d96-a659bb5295c2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.011503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.019625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.021020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.021789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.022388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.082304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 102/201] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:00.083120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.097907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.111742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.113344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.203164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e3dd-058e-4228-90ca-024594c16e25 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.232194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 100/202] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:00.232830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.245780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.246941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.247372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.247949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.249948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.251745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.253919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 103/203] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:00.254806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.264734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.343049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.355646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ca243-173e-4db3-a1b5-daf6ffd360d3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.380160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d5bb6-1ae8-4fc8-995e-d45c5c522a04 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.394325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.395625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.396346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.396844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.398512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.400420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.402900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 101/204] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:00.404334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.421784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.422738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.423386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.423670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.425120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.427021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.428940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 104/205] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:00.429534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.512677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.522711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe1d8d-c836-48cd-beff-839de085f706 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.544772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c7cf1-b429-49a2-ab01-dc8c81eb4321 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.560536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.561511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.561793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.562180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.563139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] RBAC: Adding query filter params (role.id=55a716d562db424fbfa1add5c62abd9a, scope.project.id=70324050ec23474180b5bb70f04382d0, user.id=7126ee0f823d41fbafeea61dbd855e3c) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:00.563448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.564921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.571329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 102/206] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/role_assignments?user.id=7126ee0f823d41fbafeea61dbd855e3c&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=55a716d562db424fbfa1add5c62abd9a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:00.572197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.588233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdeded-f6d9-419f-8b10-81b955e9a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.589151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdeded-f6d9-419f-8b10-81b955e9a2bb None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.589919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdeded-f6d9-419f-8b10-81b955e9a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.590259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdeded-f6d9-419f-8b10-81b955e9a2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.591733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdeded-f6d9-419f-8b10-81b955e9a2bb None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=0a28b4be4d684511a1e8a9dcf4511f94) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:00.592077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdeded-f6d9-419f-8b10-81b955e9a2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.594507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdeded-f6d9-419f-8b10-81b955e9a2bb None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.600649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.604625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 105/207] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/role_assignments?user.id=0a28b4be4d684511a1e8a9dcf4511f94&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:00.694374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efbfc9-a572-4249-8ec2-f662b6bae4c7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.734168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981574d7-0337-475c-a094-de33276f0b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.734901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981574d7-0337-475c-a094-de33276f0b6b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.735187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981574d7-0337-475c-a094-de33276f0b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.735463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981574d7-0337-475c-a094-de33276f0b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.736364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981574d7-0337-475c-a094-de33276f0b6b None admin] RBAC: Adding query filter params (role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=0a28b4be4d684511a1e8a9dcf4511f94) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:00.736826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981574d7-0337-475c-a094-de33276f0b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.738307 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981574d7-0337-475c-a094-de33276f0b6b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.745272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 103/208] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/role_assignments?user.id=0a28b4be4d684511a1e8a9dcf4511f94&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:01.636432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bdf1d-8b5e-4261-b89d-1d0b782cb9c2 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.636922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bdf1d-8b5e-4261-b89d-1d0b782cb9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.636922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bdf1d-8b5e-4261-b89d-1d0b782cb9c2 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.638527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 106/209] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:01.644010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97342-9bd1-40db-9b58-b42cf48f905a None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.644401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97342-9bd1-40db-9b58-b42cf48f905a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.644585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97342-9bd1-40db-9b58-b42cf48f905a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.661294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-05f97342-9bd1-40db-9b58-b42cf48f905a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:01.662477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-05f97342-9bd1-40db-9b58-b42cf48f905a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:01.676974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5bb1-9874-4e40-bb87-ccfef0d46fc0 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.677262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5bb1-9874-4e40-bb87-ccfef0d46fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.677541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5bb1-9874-4e40-bb87-ccfef0d46fc0 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.679038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 107/210] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:01.684717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfee66-81af-44a4-b314-8e1c1077b4ef None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.685003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfee66-81af-44a4-b314-8e1c1077b4ef None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.685280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfee66-81af-44a4-b314-8e1c1077b4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.707351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0fbfee66-81af-44a4-b314-8e1c1077b4ef None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:01.708838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0fbfee66-81af-44a4-b314-8e1c1077b4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:01.749750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 104/211] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:01.773044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f97342-9bd1-40db-9b58-b42cf48f905a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.791862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.804135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 108/212] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:01.828555 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfee66-81af-44a4-b314-8e1c1077b4ef None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.898425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.900613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f97342-9bd1-40db-9b58-b42cf48f905a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.946357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfee66-81af-44a4-b314-8e1c1077b4ef None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.950621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.951685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.952218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.952712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.968764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:01.979972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:01.980909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:01.982319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 105/213] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:01.992240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.992386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.992386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.992386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.992386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.993524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.995193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.997703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 109/214] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.004879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.065875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.089758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1b7b1-87ab-4be6-9d8a-dbaf8e24e680 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.121993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae01781-cb6c-4939-b0ad-c7e5597a777f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.123911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.124661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.125015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.125361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.126186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:02.126569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.128054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.141731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 106/215] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:02.142653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.157875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.158572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.158856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.159129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.159871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.161607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.169578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.189790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 110/216] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:02 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:02.272823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9c899-68ff-4064-a12b-ebf2ad384013 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.312716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.313613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.313894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.314170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] PATH_INFO: `/v3/users/fdfdd294a9764b418c6bc3a11d572922` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.324244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.331633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=fdfdd294a9764b418c6bc3a11d572922)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.333167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.335481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 107/217] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/users/fdfdd294a9764b418c6bc3a11d572922 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.340780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.448947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d92751-9471-4e0e-ae00-e65eb810c0c2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.485228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.486103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.486390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.486663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.555940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.569266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.570771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.686362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 111/218] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:10:02.691979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.799860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2d5d-65f8-4ba4-a8e6-a1e23d76a3b5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.836193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2271199-9eca-4d56-b153-575027e5957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.837987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2271199-9eca-4d56-b153-575027e5957d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.838474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2271199-9eca-4d56-b153-575027e5957d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.838970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2271199-9eca-4d56-b153-575027e5957d None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.845054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2271199-9eca-4d56-b153-575027e5957d None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.846910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2271199-9eca-4d56-b153-575027e5957d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.849048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 108/219] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.855851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.966920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.977466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45cf35-16cf-4f74-88ba-2468eeefa5e3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.030635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad65797-9faf-4713-86fc-d3d0e1f732e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.031650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65797-9faf-4713-86fc-d3d0e1f732e5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.032193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65797-9faf-4713-86fc-d3d0e1f732e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.032675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65797-9faf-4713-86fc-d3d0e1f732e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.033729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65797-9faf-4713-86fc-d3d0e1f732e5 None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=fdfdd294a9764b418c6bc3a11d572922) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:03.034246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65797-9faf-4713-86fc-d3d0e1f732e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:03.036001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65797-9faf-4713-86fc-d3d0e1f732e5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:03.048275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 112/220] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/role_assignments?user.id=fdfdd294a9764b418c6bc3a11d572922&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:03.061477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07e29-a24e-46e9-8753-cfc498d3cff6 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.061761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07e29-a24e-46e9-8753-cfc498d3cff6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.062043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07e29-a24e-46e9-8753-cfc498d3cff6 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.063591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 109/221] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:03.069068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c78af-485b-41c1-9e2e-546c26e4807e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.069363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c78af-485b-41c1-9e2e-546c26e4807e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.069647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c78af-485b-41c1-9e2e-546c26e4807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.094083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b93c78af-485b-41c1-9e2e-546c26e4807e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:03.095650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b93c78af-485b-41c1-9e2e-546c26e4807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:03.117875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6676e0-bf8b-4d79-b987-c46c23fa914a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.118208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6676e0-bf8b-4d79-b987-c46c23fa914a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.118451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6676e0-bf8b-4d79-b987-c46c23fa914a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.120157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 110/222] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:03.125338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992ab8b-4a1e-4772-95fd-635d34e26184 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.125625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992ab8b-4a1e-4772-95fd-635d34e26184 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.125905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992ab8b-4a1e-4772-95fd-635d34e26184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.152277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f992ab8b-4a1e-4772-95fd-635d34e26184 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:03.154759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f992ab8b-4a1e-4772-95fd-635d34e26184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:03.200915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 113/223] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:03.225015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c78af-485b-41c1-9e2e-546c26e4807e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.300738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:03.320263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 111/224] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:03.350744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992ab8b-4a1e-4772-95fd-635d34e26184 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.360970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:03.401668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c78af-485b-41c1-9e2e-546c26e4807e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.470280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.471092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.471367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.471637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.492567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:03.500394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:03.501302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:03.502106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 114/225] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:03.512945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.522029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992ab8b-4a1e-4772-95fd-635d34e26184 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.591834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.592652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.592929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.593199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.607075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:03.617200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:03.619462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:03.620694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 112/226] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:03.627950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.692355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:03.713627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079feca-bc0a-49fb-badc-f8aef24c3481 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.745637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412730e1-013f-4cb8-80f7-c86af8621e45 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.782821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.783490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.783769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.784190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.784949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:03.785256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:03.786645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:03.789508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.790183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.790497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.790761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.791525 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:03.791978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:03.793366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:03.802538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 113/227] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:03.810769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 115/228] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:03.812853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.815741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.832482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:03.998048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7f1c1-e450-4071-8a84-91131d113e2a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:04.006265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a65cd-ebcd-46fb-8b43-779d5ba184e4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:04.036205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:04.043494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:04.047045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.052213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.052498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] PATH_INFO: `/v3/users/0a28b4be4d684511a1e8a9dcf4511f94` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.053902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] RBAC: Authorizing `identity:get_user(user_id=0a28b4be4d684511a1e8a9dcf4511f94)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:04.055653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:04.057176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:04.058118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 116/229] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/users/0a28b4be4d684511a1e8a9dcf4511f94 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:04.058788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.059274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.059759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] PATH_INFO: `/v3/users/0a28b4be4d684511a1e8a9dcf4511f94` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.061282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] RBAC: Authorizing `identity:get_user(user_id=0a28b4be4d684511a1e8a9dcf4511f94)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:04.062972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:04.063167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:04.065147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 114/230] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/users/0a28b4be4d684511a1e8a9dcf4511f94 => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:04.070818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:04.210854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:04.241448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721db5d6-e422-40fb-9496-c48d8aa0add0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:04.281148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:04.282008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.282291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.282558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.284864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3006d-ecfd-472f-95a2-ec18603e3bcb None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:04.286968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:04.288521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:04.290229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 117/231] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:04 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:04.305451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:04.363198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:04.364543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.365031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.365506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.366952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:04.376127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:04.378220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 115/232] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:04 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:04.383700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:04.435903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656c41-caa7-412c-bcfb-c6ec10cce015 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:04.474524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc6dc-82a2-460d-9d07-4db78c992b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:04.475333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc6dc-82a2-460d-9d07-4db78c992b7d None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.475617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc6dc-82a2-460d-9d07-4db78c992b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.476314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc6dc-82a2-460d-9d07-4db78c992b7d None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/0a28b4be4d684511a1e8a9dcf4511f94/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.484497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc6dc-82a2-460d-9d07-4db78c992b7d None admin] RBAC: Authorizing `identity:create_grant(project_id=49a0475e9f4540ef95098d15f52a6b28, user_id=0a28b4be4d684511a1e8a9dcf4511f94, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:04.485963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc6dc-82a2-460d-9d07-4db78c992b7d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:04.506499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:04.516275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 118/233] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:04 2022] PUT /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/0a28b4be4d684511a1e8a9dcf4511f94/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:10:04.559888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72aea3-7b0e-407a-9086-ff50fada2a44 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:04.597191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c333f-e417-470e-9905-6564e3dbfccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:04.598155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c333f-e417-470e-9905-6564e3dbfccc None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.598430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c333f-e417-470e-9905-6564e3dbfccc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.598702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c333f-e417-470e-9905-6564e3dbfccc None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/0a28b4be4d684511a1e8a9dcf4511f94/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.601029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c333f-e417-470e-9905-6564e3dbfccc None admin] RBAC: Authorizing `identity:create_grant(project_id=49a0475e9f4540ef95098d15f52a6b28, user_id=0a28b4be4d684511a1e8a9dcf4511f94, role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:04.602655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c333f-e417-470e-9905-6564e3dbfccc None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:04.619157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 116/234] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:04 2022] PUT /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/0a28b4be4d684511a1e8a9dcf4511f94/roles/cabb92ea03d84bd399329e57ac452f9c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:05.167808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093272be-3598-4518-a88b-60570de961c4 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.168342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093272be-3598-4518-a88b-60570de961c4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.168417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093272be-3598-4518-a88b-60570de961c4 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.169904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 119/235] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:05.185976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dda9e-a0ec-4af4-a1b8-d5a3ab05772c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.186435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dda9e-a0ec-4af4-a1b8-d5a3ab05772c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.186435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dda9e-a0ec-4af4-a1b8-d5a3ab05772c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.209143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0f9dda9e-a0ec-4af4-a1b8-d5a3ab05772c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:05.210654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0f9dda9e-a0ec-4af4-a1b8-d5a3ab05772c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:05.266670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2762c-9c21-4c2b-8a24-92589f75e18e None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.266977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2762c-9c21-4c2b-8a24-92589f75e18e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.267241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2762c-9c21-4c2b-8a24-92589f75e18e None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.268905 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 120/236] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:05.284311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b013b-13fa-45f4-92f1-53d3cbde7bfc None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.284937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b013b-13fa-45f4-92f1-53d3cbde7bfc None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.285421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b013b-13fa-45f4-92f1-53d3cbde7bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.292639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.311289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 117/237] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:05.312454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fb5b013b-13fa-45f4-92f1-53d3cbde7bfc None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:05.313875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fb5b013b-13fa-45f4-92f1-53d3cbde7bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:05.341730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dda9e-a0ec-4af4-a1b8-d5a3ab05772c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.385730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.413552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 121/238] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:05.440481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b013b-13fa-45f4-92f1-53d3cbde7bfc None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.475398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dda9e-a0ec-4af4-a1b8-d5a3ab05772c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.512904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.513943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.514324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.514702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.536396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:05.545022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:05.545877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:05.546677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 118/239] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:05.557001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.583229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b013b-13fa-45f4-92f1-53d3cbde7bfc None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.652821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.654888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.660225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.660516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.661830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:05.663321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:05.664983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 122/240] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:05.670365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.684174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.711601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb6bfe-1389-4774-b246-74d3f8d1ad67 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.714421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.748015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe654804-d11e-4f34-be76-5182b06cc8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.748945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe654804-d11e-4f34-be76-5182b06cc8dc None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.749368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe654804-d11e-4f34-be76-5182b06cc8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.749883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe654804-d11e-4f34-be76-5182b06cc8dc None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.750869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe654804-d11e-4f34-be76-5182b06cc8dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:05.751400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe654804-d11e-4f34-be76-5182b06cc8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:05.753028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe654804-d11e-4f34-be76-5182b06cc8dc None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:05.762496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 119/241] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:05.768488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59558c53-53e0-41f1-877f-08cf27106d4b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.769110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59558c53-53e0-41f1-877f-08cf27106d4b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.769608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59558c53-53e0-41f1-877f-08cf27106d4b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.771530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 120/242] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:05.772707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59558c53-53e0-41f1-877f-08cf27106d4b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.785573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2613a-b2ac-4e1f-885b-b35ff1289fd8 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.822454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dc351-ceeb-43fa-b90b-63ad62d1d928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.823137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dc351-ceeb-43fa-b90b-63ad62d1d928 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.823479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dc351-ceeb-43fa-b90b-63ad62d1d928 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.823677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dc351-ceeb-43fa-b90b-63ad62d1d928 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.824435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dc351-ceeb-43fa-b90b-63ad62d1d928 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:05.825976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dc351-ceeb-43fa-b90b-63ad62d1d928 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:05.851409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 123/243] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:05 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:05.853119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47482c30-ac07-4b4e-8307-b7b479cd8cf4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.853411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47482c30-ac07-4b4e-8307-b7b479cd8cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.853703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47482c30-ac07-4b4e-8307-b7b479cd8cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.876020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-47482c30-ac07-4b4e-8307-b7b479cd8cf4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:05.877813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-47482c30-ac07-4b4e-8307-b7b479cd8cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:05.894113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59558c53-53e0-41f1-877f-08cf27106d4b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.956589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.957644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.958026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.958399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] PATH_INFO: `/v3/users/7e0709b62d014988bcc98fa86de24ea1` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.959870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.978840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 124/244] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:05.980549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0709b62d014988bcc98fa86de24ea1)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:05.982422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:05.984970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 121/245] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/users/7e0709b62d014988bcc98fa86de24ea1 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:05.997061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47482c30-ac07-4b4e-8307-b7b479cd8cf4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.003415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.059595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.162807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47482c30-ac07-4b4e-8307-b7b479cd8cf4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.177052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb03ed0-b255-4722-a6b4-89c9eab12072 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.203299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.204063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.204347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.204627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.214315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.215124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.215413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.215695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.228536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:06.236489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:06.237359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:06.238172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 122/246] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:06.245569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.282056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.292083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.293555 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.352197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.358404 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11135-a882-4c01-875c-c1353e4d81e0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.402542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.403235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.403521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.403798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.404590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:06.404902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.406290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.417093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 123/247] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:06.423038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.434797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 125/248] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:06.440352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.549030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910412d8-64ef-414c-b5ff-2838c12baf66 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.564231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.566716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66c8bc-7e62-46b1-8c19-1402e2f7d8dd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.593618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.594549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.594838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.595113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] PATH_INFO: `/v3/users/fdfdd294a9764b418c6bc3a11d572922` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.596375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] RBAC: Authorizing `identity:get_user(user_id=fdfdd294a9764b418c6bc3a11d572922)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.597894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.599931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 124/249] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/users/fdfdd294a9764b418c6bc3a11d572922 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:06.612171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.613042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.613326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.613603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.615800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.617305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.619078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 126/250] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:06.620240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.626768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.648347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.819628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c378ed5-0c7c-459c-b74a-491286ea8a87 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.824129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a071f-4de3-4ad0-a4e8-b544267fc8ba None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.890352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944cee9-b36c-4666-927b-bd53e9b8e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.891021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944cee9-b36c-4666-927b-bd53e9b8e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.891308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944cee9-b36c-4666-927b-bd53e9b8e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.891590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944cee9-b36c-4666-927b-bd53e9b8e0c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.892483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944cee9-b36c-4666-927b-bd53e9b8e0c2 None admin] RBAC: Adding query filter params (role.id=55a716d562db424fbfa1add5c62abd9a, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=7e0709b62d014988bcc98fa86de24ea1) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:06.892797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944cee9-b36c-4666-927b-bd53e9b8e0c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.894253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944cee9-b36c-4666-927b-bd53e9b8e0c2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.896692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.897565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.897846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.898124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.899377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.900946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.902691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 125/251] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:06.908616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.914861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 127/252] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/role_assignments?user.id=7e0709b62d014988bcc98fa86de24ea1&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=55a716d562db424fbfa1add5c62abd9a => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:07.030639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.055463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e89b7-6602-4281-95aa-c47839081d5e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.110038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bab366-98b5-489e-98fa-eb54c879a1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.110939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bab366-98b5-489e-98fa-eb54c879a1d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.111225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bab366-98b5-489e-98fa-eb54c879a1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.111502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bab366-98b5-489e-98fa-eb54c879a1d2 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/fdfdd294a9764b418c6bc3a11d572922/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.117176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bab366-98b5-489e-98fa-eb54c879a1d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=49a0475e9f4540ef95098d15f52a6b28, user_id=fdfdd294a9764b418c6bc3a11d572922, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.119221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bab366-98b5-489e-98fa-eb54c879a1d2 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.157823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 126/253] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:06 2022] PUT /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/fdfdd294a9764b418c6bc3a11d572922/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:10:07.356004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0c436-5481-4812-9f1a-8f851e5dbaf8 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.356518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0c436-5481-4812-9f1a-8f851e5dbaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.356593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0c436-5481-4812-9f1a-8f851e5dbaf8 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.358296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 128/254] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:07.363859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb441ba-90cc-455e-9829-3931e66fcc0f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.364356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb441ba-90cc-455e-9829-3931e66fcc0f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.364356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb441ba-90cc-455e-9829-3931e66fcc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.385371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0fb441ba-90cc-455e-9829-3931e66fcc0f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:07.386650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0fb441ba-90cc-455e-9829-3931e66fcc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:07.474674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 127/255] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:07.510425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0c436-5481-4812-9f1a-8f851e5dbaf8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.521019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d3c85-4c82-4322-8460-e8c500881dff None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.521312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d3c85-4c82-4322-8460-e8c500881dff None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.521583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d3c85-4c82-4322-8460-e8c500881dff None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.529277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 128/256] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:07.541025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fa786-8222-4b30-b528-71b9be0b496c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.541316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fa786-8222-4b30-b528-71b9be0b496c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.541599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fa786-8222-4b30-b528-71b9be0b496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.554577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.566984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-5f9fa786-8222-4b30-b528-71b9be0b496c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:07.569317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5f9fa786-8222-4b30-b528-71b9be0b496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:07.718237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0c436-5481-4812-9f1a-8f851e5dbaf8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.733104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.749883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 129/257] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:07.785788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.786600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.786889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.787170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.789850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fa786-8222-4b30-b528-71b9be0b496c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.807685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:07.820278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:07.821165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:07.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:07.825575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 129/258] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:07.830939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.035260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.066569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fa786-8222-4b30-b528-71b9be0b496c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.081674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4eef2-810e-450f-8bd0-52077ece7721 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.119522 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.120863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.121350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.121852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.141887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:08.150697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:08.151647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:08.152493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:08.155395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 130/259] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:08.162901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.168018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.168420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.168717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.169001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.169776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:08.170095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.171514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.193772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 130/260] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:08.199910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.253699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.310717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca28365-6558-435b-b85c-9204b9d784a1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.365230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.367814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.369039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.370333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.371296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:08.376195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.377824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.391453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 131/261] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:08.401806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.457719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e263e5-5062-43ce-a221-6fecc157282c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.517032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe94048-d517-449f-81b9-2b3067ebd512 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.562425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.585134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.586320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.586889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.587397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] PATH_INFO: `/v3/users/0a28b4be4d684511a1e8a9dcf4511f94` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.588837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] RBAC: Authorizing `identity:get_user(user_id=0a28b4be4d684511a1e8a9dcf4511f94)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.590778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.592914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 131/262] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/users/0a28b4be4d684511a1e8a9dcf4511f94 => generated 283 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:08.595840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.595840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.595840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.595840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] PATH_INFO: `/v3/users/0a28b4be4d684511a1e8a9dcf4511f94` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.596739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] RBAC: Authorizing `identity:get_user(user_id=0a28b4be4d684511a1e8a9dcf4511f94)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.598178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.599871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 132/263] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/users/0a28b4be4d684511a1e8a9dcf4511f94 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:08.605703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.605967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.631265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.854545 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbf5d3-7c9a-4f5d-9be5-f81e54949710 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.854741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74f4c4-be2f-4418-8b9c-1a8244dec0d3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.927400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.928186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.928477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.928756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.930599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bda14-d460-47bc-a2ff-f8537c357c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.931546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bda14-d460-47bc-a2ff-f8537c357c45 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.932070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bda14-d460-47bc-a2ff-f8537c357c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.932574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bda14-d460-47bc-a2ff-f8537c357c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.031950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.045547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.047022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.133050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bda14-d460-47bc-a2ff-f8537c357c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.134512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bda14-d460-47bc-a2ff-f8537c357c45 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.160171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.200956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 133/264] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:09.206798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.339861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.352240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ff3bd-b0e3-4ab1-98f8-c8dc9f4b73c0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.400211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 132/265] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:09.401488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed573c-7fb9-4bf7-b078-a01cd81c523b None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.401780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed573c-7fb9-4bf7-b078-a01cd81c523b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.402061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed573c-7fb9-4bf7-b078-a01cd81c523b None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.403344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 133/266] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:09.411142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed573c-7fb9-4bf7-b078-a01cd81c523b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.415917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da0dea-3540-4a08-9da8-23ac28a91968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.416738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da0dea-3540-4a08-9da8-23ac28a91968 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.417023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da0dea-3540-4a08-9da8-23ac28a91968 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.417302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da0dea-3540-4a08-9da8-23ac28a91968 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.418554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da0dea-3540-4a08-9da8-23ac28a91968 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.420048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da0dea-3540-4a08-9da8-23ac28a91968 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.421764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 134/267] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.427859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3092b-b166-43f6-97dc-eb94103af43c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.428158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3092b-b166-43f6-97dc-eb94103af43c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.428446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3092b-b166-43f6-97dc-eb94103af43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.461494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c8e3092b-b166-43f6-97dc-eb94103af43c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:09.463241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c8e3092b-b166-43f6-97dc-eb94103af43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:09.584710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 135/268] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:09.585580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3092b-b166-43f6-97dc-eb94103af43c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.612538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed573c-7fb9-4bf7-b078-a01cd81c523b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.631857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.686578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.702427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3092b-b166-43f6-97dc-eb94103af43c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.711263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.712767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.713049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.713322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.714623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.716376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.718539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 134/269] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.719321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.750522 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.751238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.751529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.751806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.754345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] RBAC: Adding query filter params (role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=0a28b4be4d684511a1e8a9dcf4511f94) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.754665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.758039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.770551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 136/270] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/role_assignments?user.id=0a28b4be4d684511a1e8a9dcf4511f94&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.771380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.892548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d017482-44e8-4166-8aa3-4b5f2e651866 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.926197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cce1e-b35b-4db9-affd-2f4e7a4293d4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.944862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992df79-cfdd-416f-8d12-54de07757b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.945985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992df79-cfdd-416f-8d12-54de07757b67 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.947013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992df79-cfdd-416f-8d12-54de07757b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.947491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992df79-cfdd-416f-8d12-54de07757b67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.958972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992df79-cfdd-416f-8d12-54de07757b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:09.964613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.965346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.965637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.966211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.967083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=0a28b4be4d684511a1e8a9dcf4511f94) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.967399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.968933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2992df79-cfdd-416f-8d12-54de07757b67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:09.969699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:09.970559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:09.971358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:09.978842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 135/271] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:09.978842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50cffd-62e2-47a6-9c5a-f963ce47493d None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.978842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50cffd-62e2-47a6-9c5a-f963ce47493d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.979536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 137/272] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/role_assignments?user.id=0a28b4be4d684511a1e8a9dcf4511f94&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.979763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50cffd-62e2-47a6-9c5a-f963ce47493d None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.980706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.982531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 136/273] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:09.989802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21d60-f322-4b03-8812-25c1ce575714 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.990140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21d60-f322-4b03-8812-25c1ce575714 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.990369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21d60-f322-4b03-8812-25c1ce575714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.028256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.031101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e2e21d60-f322-4b03-8812-25c1ce575714 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:10.032489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e2e21d60-f322-4b03-8812-25c1ce575714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:10.103686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.172983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 137/274] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:10.203457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16938121-da99-4b3b-8bd0-f098dcdecc82 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.203752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16938121-da99-4b3b-8bd0-f098dcdecc82 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.203967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16938121-da99-4b3b-8bd0-f098dcdecc82 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.206085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 138/275] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:10.209677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16938121-da99-4b3b-8bd0-f098dcdecc82 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.224959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc40d0a-f7dd-4c80-810c-1b2216de5f66 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.275459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce917c0-c479-4d23-80a4-b98b8ff55d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.276482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce917c0-c479-4d23-80a4-b98b8ff55d06 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.276773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce917c0-c479-4d23-80a4-b98b8ff55d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.277060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce917c0-c479-4d23-80a4-b98b8ff55d06 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.277883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce917c0-c479-4d23-80a4-b98b8ff55d06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:10.278203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce917c0-c479-4d23-80a4-b98b8ff55d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.280159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce917c0-c479-4d23-80a4-b98b8ff55d06 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.290249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 138/276] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.291499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1754f0b-569f-49b8-9387-d72a66143663 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.291794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1754f0b-569f-49b8-9387-d72a66143663 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.292108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1754f0b-569f-49b8-9387-d72a66143663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.311923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a1754f0b-569f-49b8-9387-d72a66143663 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:10.313115 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a1754f0b-569f-49b8-9387-d72a66143663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:10.347219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16938121-da99-4b3b-8bd0-f098dcdecc82 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.389452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.391227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.391505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.391779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.393074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.394639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.396644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 139/277] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.397435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.403126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 139/278] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:10.405303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1754f0b-569f-49b8-9387-d72a66143663 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.416634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.437280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.501527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5faf1-7957-4c3e-a915-987161a9575e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.506471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1754f0b-569f-49b8-9387-d72a66143663 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.533887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77186fa0-fca2-4476-a678-c76785a20a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.534751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77186fa0-fca2-4476-a678-c76785a20a0c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.535036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77186fa0-fca2-4476-a678-c76785a20a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.535305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77186fa0-fca2-4476-a678-c76785a20a0c None admin] PATH_INFO: `/v3/users/67ce21490bfe45889c7acfd811193256` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.540213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.540876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.541158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.541431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.542137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.543598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.551207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77186fa0-fca2-4476-a678-c76785a20a0c None admin] RBAC: Authorizing `identity:get_user(user_id=67ce21490bfe45889c7acfd811193256)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.552669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77186fa0-fca2-4476-a678-c76785a20a0c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.554519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 140/279] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/users/67ce21490bfe45889c7acfd811193256 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.555536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94d6c0-48b5-4151-806a-df42b398a553 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.556343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94d6c0-48b5-4151-806a-df42b398a553 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.556604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94d6c0-48b5-4151-806a-df42b398a553 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.557827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 141/280] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:10.558510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94d6c0-48b5-4151-806a-df42b398a553 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.569983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 140/281] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:10 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:10.570806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.641786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.676303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.685976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94d6c0-48b5-4151-806a-df42b398a553 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.690282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15226-5e75-41b8-bb94-3bdb75c62531 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.725553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.726417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.726703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.726978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.729706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed54df-dc90-4570-9c19-461cfc218ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.730377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed54df-dc90-4570-9c19-461cfc218ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.730652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed54df-dc90-4570-9c19-461cfc218ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.731126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed54df-dc90-4570-9c19-461cfc218ec9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.731772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed54df-dc90-4570-9c19-461cfc218ec9 None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, user.id=67ce21490bfe45889c7acfd811193256, scope.system=all) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:10.732116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed54df-dc90-4570-9c19-461cfc218ec9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.733622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed54df-dc90-4570-9c19-461cfc218ec9 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.739641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 141/282] 10.222.0.62 () {56 vars in 1308 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/role_assignments?user.id=67ce21490bfe45889c7acfd811193256&scope.system=all&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.741055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d08bcc-a1bf-426b-902d-662579fb4152 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.741349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d08bcc-a1bf-426b-902d-662579fb4152 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.741627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d08bcc-a1bf-426b-902d-662579fb4152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.745086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:10.755786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:10.756767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:10.757547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 142/283] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:10.761678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a9d08bcc-a1bf-426b-902d-662579fb4152 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:10.764278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a9d08bcc-a1bf-426b-902d-662579fb4152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:10.764506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.854256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 142/284] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:10.877334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d08bcc-a1bf-426b-902d-662579fb4152 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.897330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57016e3-b71d-40d6-b4fe-a2585afdf363 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.946030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.950033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.950925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.951191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.951470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.952970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] RBAC: Adding query filter params (name=member) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:10.953285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.954685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.965782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 143/285] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.973656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.000243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.004123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d08bcc-a1bf-426b-902d-662579fb4152 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.048713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.049443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.049726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.050140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.064616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:11.073950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:11.074857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:11.075649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:11.079989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 143/286] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:11.081501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.097321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfcefe-1be5-4aa5-a967-de1648d93003 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.135108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.136131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.136411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.136686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] PATH_INFO: `/v3/users/fdfdd294a9764b418c6bc3a11d572922` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.137914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] RBAC: Authorizing `identity:get_user(user_id=fdfdd294a9764b418c6bc3a11d572922)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.139599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:11.141661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 144/287] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/users/fdfdd294a9764b418c6bc3a11d572922 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:11.157324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.188951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62923eb-3a74-4cdc-bb6f-87293a5303a2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.206620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.223797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.224497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.224778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.225056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.225811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:11.226114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.227570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:11.237386 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 144/288] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:11.242928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.302266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.334281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c5c58-aaf9-41b2-baa3-68a45e838689 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.350867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5986ce8-dc73-4057-9d32-228fbbb79d47 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.394913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.396904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.397749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.398733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] PATH_INFO: `/v3/users/7e0709b62d014988bcc98fa86de24ea1` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.399572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.401036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.401929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] RBAC: Authorizing `identity:get_user(user_id=7e0709b62d014988bcc98fa86de24ea1)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.402413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.404103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:11.404615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.407092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 145/289] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/users/7e0709b62d014988bcc98fa86de24ea1 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:11.412712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.503953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.542529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.552508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:11.572510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34ccb9-5c20-4c19-a322-2841caec982d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.616803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.619122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.619838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.621009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.623863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.625606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:11.627676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 146/290] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:11.636722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.722326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.754460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 145/291] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:10:11.760598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.778851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac5dc4-609b-42cc-b828-664274829484 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.807907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.842876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.844144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.844659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.845137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/7e0709b62d014988bcc98fa86de24ea1/roles/55a716d562db424fbfa1add5c62abd9a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.848553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] RBAC: Authorizing `identity:create_grant(project_id=49a0475e9f4540ef95098d15f52a6b28, user_id=7e0709b62d014988bcc98fa86de24ea1, role_id=55a716d562db424fbfa1add5c62abd9a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.850241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:11.876039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 147/292] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:11 2022] PUT /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28/users/7e0709b62d014988bcc98fa86de24ea1/roles/55a716d562db424fbfa1add5c62abd9a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:11.939611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e529c-d38d-4b31-8fd2-fd1d6c649be1 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.991113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21abe5-1622-4b35-9985-a8fb1aa8ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.992369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21abe5-1622-4b35-9985-a8fb1aa8ff12 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.992966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21abe5-1622-4b35-9985-a8fb1aa8ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.994013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21abe5-1622-4b35-9985-a8fb1aa8ff12 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.996650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21abe5-1622-4b35-9985-a8fb1aa8ff12 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.998139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21abe5-1622-4b35-9985-a8fb1aa8ff12 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.000210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 146/293] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:12.006211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.170766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34eee8-f1e3-4299-84ca-c2202b0e1996 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.211940 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25850c69-1daf-434d-942a-814143859445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.212604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25850c69-1daf-434d-942a-814143859445 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.212884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25850c69-1daf-434d-942a-814143859445 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.213153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25850c69-1daf-434d-942a-814143859445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.213995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25850c69-1daf-434d-942a-814143859445 None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=fdfdd294a9764b418c6bc3a11d572922) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:12.214306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25850c69-1daf-434d-942a-814143859445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.215745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25850c69-1daf-434d-942a-814143859445 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.230395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 148/294] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/role_assignments?user.id=fdfdd294a9764b418c6bc3a11d572922&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:13.133057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc954190-2870-4453-8d6e-98d26bf5432d None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.134110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc954190-2870-4453-8d6e-98d26bf5432d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.134213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc954190-2870-4453-8d6e-98d26bf5432d None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.135530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 147/295] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:13.141399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c797c8-4005-4cf2-9bff-896206a623ef None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.141911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c797c8-4005-4cf2-9bff-896206a623ef None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.142174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c797c8-4005-4cf2-9bff-896206a623ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.160602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-40c797c8-4005-4cf2-9bff-896206a623ef None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:13.161804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-40c797c8-4005-4cf2-9bff-896206a623ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:13.197890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.233681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 149/296] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:13.257176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc954190-2870-4453-8d6e-98d26bf5432d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.320966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.341300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57246bea-d9af-4f36-ac05-9155203f0fa6 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.341928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57246bea-d9af-4f36-ac05-9155203f0fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.342424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57246bea-d9af-4f36-ac05-9155203f0fa6 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.344175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 150/297] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:13.348971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04d41-cdad-4c79-8ead-2bc3fb3d7ff4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.349755 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04d41-cdad-4c79-8ead-2bc3fb3d7ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.350238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04d41-cdad-4c79-8ead-2bc3fb3d7ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.365860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc954190-2870-4453-8d6e-98d26bf5432d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.371914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a5d04d41-cdad-4c79-8ead-2bc3fb3d7ff4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:13.373317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a5d04d41-cdad-4c79-8ead-2bc3fb3d7ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:13.402522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.403362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.403847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.404266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.415355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:13.423260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.424209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:13.424993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:13.427515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 148/298] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.434205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.461246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 151/299] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:13.468112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42069-392a-40d9-ab5a-34797272d7d2 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.468404 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42069-392a-40d9-ab5a-34797272d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.468684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42069-392a-40d9-ab5a-34797272d7d2 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.473721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 152/300] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:13.475314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a9b9d-d452-47e5-92bb-e744fa33d0b8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.475607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a9b9d-d452-47e5-92bb-e744fa33d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.475924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a9b9d-d452-47e5-92bb-e744fa33d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.492140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a97a9b9d-d452-47e5-92bb-e744fa33d0b8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:13.493836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a97a9b9d-d452-47e5-92bb-e744fa33d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:13.547787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006052f9-6d65-48f8-aace-251b708c5e7e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.561234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.565597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 153/301] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:13.572740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a9b9d-d452-47e5-92bb-e744fa33d0b8 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.584784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.585454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.585740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.586014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.586824 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:13.587438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.588819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.597057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 149/302] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:13.597843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.613089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.686252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a9b9d-d452-47e5-92bb-e744fa33d0b8 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.706229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8df008-fcda-4821-96ce-11e9837b34f2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.721571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.722447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.722736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.723065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.724388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.725843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.728001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 154/303] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:13.728760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.748346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.749149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.749437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.749714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.760745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:13.768815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.769705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 150/304] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.770934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.807330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.830158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.833980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449390a-c6bb-4e8c-854c-bdb4a929ffbd None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.871481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.872381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.872672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.872946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] PATH_INFO: `/v3/users/b8c82c515fa94669b3112352d15deb2a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.878191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fdb5e-fa0c-4ec0-9fc6-fa8078173505 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.887463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] RBAC: Authorizing `identity:get_user(user_id=b8c82c515fa94669b3112352d15deb2a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.889035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.890999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 155/305] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/users/b8c82c515fa94669b3112352d15deb2a => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:13.891580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.924934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.925614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.925900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.926177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] PATH_INFO: `/v3/groups` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.926883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.928386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.948427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 151/306] 10.222.0.62 () {60 vars in 1134 bytes} [Fri May 27 03:10:13 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:13.949255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.029177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f895f63-3a28-40e8-ab3d-25c13ec530a6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.074556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.075528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.075851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.076124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.076896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.077207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.079471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.092482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 156/307] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.093549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.101136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.107622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7a4ed-a95c-454b-b94d-ab2f912be04f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.128167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.148455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.149159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.149464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.149740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.150715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] RBAC: Adding query filter params (role.id=55a716d562db424fbfa1add5c62abd9a, user.id=b8c82c515fa94669b3112352d15deb2a, scope.system=all) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.151032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.152515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.159166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 152/308] 10.222.0.62 () {56 vars in 1308 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/role_assignments?user.id=b8c82c515fa94669b3112352d15deb2a&scope.system=all&role.id=55a716d562db424fbfa1add5c62abd9a => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.232594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd6e1-6a97-4683-b776-8215022bd233 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.272841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492bf6b-cbc7-48cc-b3bd-48d738bfff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.274015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492bf6b-cbc7-48cc-b3bd-48d738bfff27 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.274489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492bf6b-cbc7-48cc-b3bd-48d738bfff27 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.274943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492bf6b-cbc7-48cc-b3bd-48d738bfff27 None admin] PATH_INFO: `/v3/users/67ce21490bfe45889c7acfd811193256` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.276434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492bf6b-cbc7-48cc-b3bd-48d738bfff27 None admin] RBAC: Authorizing `identity:get_user(user_id=67ce21490bfe45889c7acfd811193256)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.278095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492bf6b-cbc7-48cc-b3bd-48d738bfff27 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.280374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 157/309] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/users/67ce21490bfe45889c7acfd811193256 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:14.286416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.386228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde2a0-345a-4425-8d2c-9d08c99df83d None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.386553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde2a0-345a-4425-8d2c-9d08c99df83d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.386790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde2a0-345a-4425-8d2c-9d08c99df83d None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.388446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 158/310] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:14.395044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdc2e9-8eac-44c9-8af4-3990cc99f177 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.395582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdc2e9-8eac-44c9-8af4-3990cc99f177 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.396077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdc2e9-8eac-44c9-8af4-3990cc99f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.401872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f0193-15f0-4c10-9b0d-d0b0080051d2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.414091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d5bdc2e9-8eac-44c9-8af4-3990cc99f177 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:14.416218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d5bdc2e9-8eac-44c9-8af4-3990cc99f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:14.437422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.438295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.438579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.438857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] PATH_INFO: `/v3/system/users/67ce21490bfe45889c7acfd811193256/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.440393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=67ce21490bfe45889c7acfd811193256, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.441783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.451594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 153/311] 10.222.0.62 () {58 vars in 1334 bytes} [Fri May 27 03:10:14 2022] PUT /identity/v3/system/users/67ce21490bfe45889c7acfd811193256/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:14.502921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 159/312] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:14.529009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.558813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.648329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d9f9-7810-465d-8d41-783070a7462f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.688551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.689362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.689645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.689920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.701432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:14.709275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:14.710193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:14.710973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 154/313] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:14.719566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdc2e9-8eac-44c9-8af4-3990cc99f177 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.740429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.826762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdc2e9-8eac-44c9-8af4-3990cc99f177 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.861191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.861870 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.862156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.862437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.863194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.863506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.864925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.874292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 160/314] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.880974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.970749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.990141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8524e9-5097-46cb-927b-118567f4ae0d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.026087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.026962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.027244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.027518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] PATH_INFO: `/v3/users/7e0709b62d014988bcc98fa86de24ea1` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.028865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] RBAC: Authorizing `identity:get_user(user_id=7e0709b62d014988bcc98fa86de24ea1)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.030323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.032256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 155/315] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/users/7e0709b62d014988bcc98fa86de24ea1 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:15.038161 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.118503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.140923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ecedc-b8bd-4de5-a231-ab5a43967ed0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.174733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.175379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.175659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.176037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.247448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.248959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.336575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.362159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 161/316] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:15.363089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.465127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19248fc4-6f18-4655-85d9-cab36afbaffd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.499448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d5b3d-b6ea-48ec-a99f-2093ca1fd82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.500337 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d5b3d-b6ea-48ec-a99f-2093ca1fd82b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.500986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d5b3d-b6ea-48ec-a99f-2093ca1fd82b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.501264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d5b3d-b6ea-48ec-a99f-2093ca1fd82b None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.502440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d5b3d-b6ea-48ec-a99f-2093ca1fd82b None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.503913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d5b3d-b6ea-48ec-a99f-2093ca1fd82b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.505903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 156/317] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:15.511504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.611668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096b61-55a9-426f-aace-d216fdeb808c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.650808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d783b4-52e7-4de5-9dd1-dab72ff2e1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.651897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d783b4-52e7-4de5-9dd1-dab72ff2e1bc None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.652384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d783b4-52e7-4de5-9dd1-dab72ff2e1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.652858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d783b4-52e7-4de5-9dd1-dab72ff2e1bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.653907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d783b4-52e7-4de5-9dd1-dab72ff2e1bc None admin] RBAC: Adding query filter params (role.id=55a716d562db424fbfa1add5c62abd9a, scope.project.id=49a0475e9f4540ef95098d15f52a6b28, user.id=7e0709b62d014988bcc98fa86de24ea1) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:15.654425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d783b4-52e7-4de5-9dd1-dab72ff2e1bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.656115 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d783b4-52e7-4de5-9dd1-dab72ff2e1bc None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.663098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 162/318] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/role_assignments?user.id=7e0709b62d014988bcc98fa86de24ea1&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=55a716d562db424fbfa1add5c62abd9a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:16.309017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003d82a-8c16-46ba-9a24-c34bbef82041 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.309354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003d82a-8c16-46ba-9a24-c34bbef82041 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.309545 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003d82a-8c16-46ba-9a24-c34bbef82041 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.310901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 157/319] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:16.316110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46251f51-929c-4c37-9304-71da5a218737 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.316562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46251f51-929c-4c37-9304-71da5a218737 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.316737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46251f51-929c-4c37-9304-71da5a218737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.333626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-46251f51-929c-4c37-9304-71da5a218737 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:16.334798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-46251f51-929c-4c37-9304-71da5a218737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:16.370013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:16.401746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 163/320] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:16.423524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003d82a-8c16-46ba-9a24-c34bbef82041 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.443077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:16.473863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267691-21ea-456c-8e91-b8804dadf599 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.474158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267691-21ea-456c-8e91-b8804dadf599 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.474442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267691-21ea-456c-8e91-b8804dadf599 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.475971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 164/321] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:16.480658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e95cec-2a4a-4560-a5cd-28c4f585e116 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.480946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e95cec-2a4a-4560-a5cd-28c4f585e116 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.481224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e95cec-2a4a-4560-a5cd-28c4f585e116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.498176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f0e95cec-2a4a-4560-a5cd-28c4f585e116 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:16.499341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f0e95cec-2a4a-4560-a5cd-28c4f585e116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:16.532861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003d82a-8c16-46ba-9a24-c34bbef82041 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:16.568151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 165/322] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:16.573267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:16.574291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.574772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.575260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.576657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:16.578343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:16.580475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 158/323] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:16.586017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e95cec-2a4a-4560-a5cd-28c4f585e116 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.591285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.690841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:16.700126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e95cec-2a4a-4560-a5cd-28c4f585e116 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:16.701677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:16.708742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d62a4a-e498-4f73-a09b-fe6bc8e1dcf8 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:16.740954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:16.741624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.741912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.742191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] PATH_INFO: `/v3/groups` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.743525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:16.744844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bdb7a-4038-48ee-b047-3955720bb67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:16.745035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:16.745637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdb7a-4038-48ee-b047-3955720bb67d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.745916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdb7a-4038-48ee-b047-3955720bb67d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.746190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdb7a-4038-48ee-b047-3955720bb67d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.757258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bdb7a-4038-48ee-b047-3955720bb67d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:16.761443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 166/324] 10.222.0.62 () {60 vars in 1134 bytes} [Fri May 27 03:10:16 2022] POST /identity/v3/groups => generated 226 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-916bdb7a-4038-48ee-b047-3955720bb67d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:16.765148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:16.766084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:16.766907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 159/325] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:16.772147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.785769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1f2a-e59d-48a0-adbd-cc2c29dcf6fa None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.786077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1f2a-e59d-48a0-adbd-cc2c29dcf6fa None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.786323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1f2a-e59d-48a0-adbd-cc2c29dcf6fa None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.787739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 160/326] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:16.794092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3692-8ac9-4a19-b5df-e34a7090da37 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.794384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3692-8ac9-4a19-b5df-e34a7090da37 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.794666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f3692-8ac9-4a19-b5df-e34a7090da37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.814666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d51f3692-8ac9-4a19-b5df-e34a7090da37 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:16.816197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d51f3692-8ac9-4a19-b5df-e34a7090da37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:16.887116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7200-22c3-4723-aa62-b63827b4d79f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:16.891859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 161/327] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:16.912765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f3692-8ac9-4a19-b5df-e34a7090da37 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.927145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273755af-56db-4444-afca-83194c16f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:16.927858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273755af-56db-4444-afca-83194c16f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.928169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273755af-56db-4444-afca-83194c16f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.928443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273755af-56db-4444-afca-83194c16f5aa None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.929254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273755af-56db-4444-afca-83194c16f5aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:16.929535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273755af-56db-4444-afca-83194c16f5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:16.931578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273755af-56db-4444-afca-83194c16f5aa None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:16.942619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 167/328] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:16.948104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273755af-56db-4444-afca-83194c16f5aa None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.982316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:17.004255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:17.034113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f3692-8ac9-4a19-b5df-e34a7090da37 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.076103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273755af-56db-4444-afca-83194c16f5aa None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.081324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.082132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.082416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.082690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.096187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:17.105202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:17.106109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:17.106891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 162/329] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:17.112654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:17.119214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.120639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.120639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.120639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] PATH_INFO: `/v3/users/b8c82c515fa94669b3112352d15deb2a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.122577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] RBAC: Authorizing `identity:get_user(user_id=b8c82c515fa94669b3112352d15deb2a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:17.123291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:17.125249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 168/330] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:16 2022] GET /identity/v3/users/b8c82c515fa94669b3112352d15deb2a => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:17.130688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:17.236608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141cec4-eb8c-4f35-86c4-ee57a3b51320 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.263907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:17.275511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507a11-7aa6-4835-a151-b5ae880e83ec None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.302413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.303103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.303388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.303666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.304480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] RBAC: Adding query filter params (name=member) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:17.304793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:17.306238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:17.310054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:17.319017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 163/331] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:17.325034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:17.336199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.339299 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.340199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.341306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] PATH_INFO: `/v3/system/users/b8c82c515fa94669b3112352d15deb2a/roles/55a716d562db424fbfa1add5c62abd9a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.343443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b8c82c515fa94669b3112352d15deb2a, role_id=55a716d562db424fbfa1add5c62abd9a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:17.347001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:17.360836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 169/332] 10.222.0.62 () {58 vars in 1334 bytes} [Fri May 27 03:10:17 2022] PUT /identity/v3/system/users/b8c82c515fa94669b3112352d15deb2a/roles/55a716d562db424fbfa1add5c62abd9a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:10:17.453533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c502-9e8e-4fb9-8edf-e3a6b0515774 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.493617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea1fdb-594e-4014-83de-43a6fc055aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.494459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea1fdb-594e-4014-83de-43a6fc055aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.494779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea1fdb-594e-4014-83de-43a6fc055aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.495053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea1fdb-594e-4014-83de-43a6fc055aa6 None admin] PATH_INFO: `/v3/users/67ce21490bfe45889c7acfd811193256` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.496409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea1fdb-594e-4014-83de-43a6fc055aa6 None admin] RBAC: Authorizing `identity:get_user(user_id=67ce21490bfe45889c7acfd811193256)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:17.498014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea1fdb-594e-4014-83de-43a6fc055aa6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:17.500054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 164/333] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:17 2022] GET /identity/v3/users/67ce21490bfe45889c7acfd811193256 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:17.505685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:17.646006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3593-16e9-4319-a5e2-eeb0ac9b68b3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.684487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874efbd-ee85-4a90-9350-d66562bc46e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.685548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874efbd-ee85-4a90-9350-d66562bc46e1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.686096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874efbd-ee85-4a90-9350-d66562bc46e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.686596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874efbd-ee85-4a90-9350-d66562bc46e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.687659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874efbd-ee85-4a90-9350-d66562bc46e1 None admin] RBAC: Adding query filter params (role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, user.id=67ce21490bfe45889c7acfd811193256, scope.system=all) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:17.688221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874efbd-ee85-4a90-9350-d66562bc46e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:17.689868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874efbd-ee85-4a90-9350-d66562bc46e1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:17.704359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 170/334] 10.222.0.62 () {56 vars in 1308 bytes} [Fri May 27 03:10:17 2022] GET /identity/v3/role_assignments?user.id=67ce21490bfe45889c7acfd811193256&scope.system=all&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:19.813725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6cc00-85a5-4663-ba7a-e634b3559d83 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.816062 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a916d9-e225-478d-9e0e-b1aa1166e047 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.816913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a916d9-e225-478d-9e0e-b1aa1166e047 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.817415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6cc00-85a5-4663-ba7a-e634b3559d83 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.817921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a916d9-e225-478d-9e0e-b1aa1166e047 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.818406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6cc00-85a5-4663-ba7a-e634b3559d83 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.821916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 171/335] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:19.822332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 165/336] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:19.825417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1437-ea2a-4007-81dd-f2b1bdd2ea41 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.827050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046066-9af6-4dc7-8206-c18755d1436e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.827555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1437-ea2a-4007-81dd-f2b1bdd2ea41 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.828085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046066-9af6-4dc7-8206-c18755d1436e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.828589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1437-ea2a-4007-81dd-f2b1bdd2ea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.829072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed046066-9af6-4dc7-8206-c18755d1436e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.864970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ed046066-9af6-4dc7-8206-c18755d1436e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:19.866222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ceca1437-ea2a-4007-81dd-f2b1bdd2ea41 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:19.868209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ed046066-9af6-4dc7-8206-c18755d1436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:19.869576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ceca1437-ea2a-4007-81dd-f2b1bdd2ea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:19.891532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.027960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.042189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 172/337] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:20.043583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e201a-da4a-44ac-8cd7-688f9693f0e5 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.044602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 166/338] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:20.044720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e201a-da4a-44ac-8cd7-688f9693f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.044966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e201a-da4a-44ac-8cd7-688f9693f0e5 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.046310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 173/339] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:20.053277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e84d-4868-43e7-9f55-43a441128ba5 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.053572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e84d-4868-43e7-9f55-43a441128ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.053855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e84d-4868-43e7-9f55-43a441128ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.078153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e201a-da4a-44ac-8cd7-688f9693f0e5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.082641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d7c4e84d-4868-43e7-9f55-43a441128ba5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:20.083846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d7c4e84d-4868-43e7-9f55-43a441128ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:20.202976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 167/340] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:20.203694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e84d-4868-43e7-9f55-43a441128ba5 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.267279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e201a-da4a-44ac-8cd7-688f9693f0e5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.337942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.338775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.339063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.339341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.364866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:20.366884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e84d-4868-43e7-9f55-43a441128ba5 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:20.375672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:20.376696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:20.377536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:20.382169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 174/341] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:20.383226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.387139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.437618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.464556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.465471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.466090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.466431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.492835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:20.500758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:20.502254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 168/342] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:20.503371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.511545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a5e27-a626-4b2c-8e2a-55b72c69b7be None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.549814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.550959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.551438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.551925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.563126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:20.571881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:20.572835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:20.573682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:20.576328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 175/343] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:20.577319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.645141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.685063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e5bd-03e7-43e8-991e-47c47fa27ed7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.723078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.724085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.724780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.725292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.726225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:20.726717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.728693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.737815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 176/344] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:20.739070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.782234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9ea9e-d04b-4ce5-ba27-640e2e83e89d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.866478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc91c9-7f53-4df1-802f-310006e19ee1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.887789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.888526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.888869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.889210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.890024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:20.892085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.893700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.900262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.908517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.909501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.910029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.910480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.910600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 169/345] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:20.911511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.911704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:20.912350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.913965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.940516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 177/346] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:20.941950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.990453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.053629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92924f2d-76fd-4f6e-9fc8-1ff9a80c75e7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.060728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dcdc0-c373-4402-9b3f-0581ccaed902 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.092867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b5d87-203c-4128-8a5a-161191df5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.093528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b5d87-203c-4128-8a5a-161191df5102 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.093814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b5d87-203c-4128-8a5a-161191df5102 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.094097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b5d87-203c-4128-8a5a-161191df5102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.098886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.099751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.100200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.101582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.177049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.191388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.192948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.194117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b5d87-203c-4128-8a5a-161191df5102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.195764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b5d87-203c-4128-8a5a-161191df5102 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.255759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.319977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 170/347] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:21.321410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.335605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 178/348] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:21.335605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8b5a-1eab-43ec-b1f2-65004155bfb1 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.335605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8b5a-1eab-43ec-b1f2-65004155bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.335605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8b5a-1eab-43ec-b1f2-65004155bfb1 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.335605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 179/349] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:21.336577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa0d9d-a1eb-4bcb-80f1-3ab3095befb8 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.336926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa0d9d-a1eb-4bcb-80f1-3ab3095befb8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.337210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa0d9d-a1eb-4bcb-80f1-3ab3095befb8 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.338440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 180/350] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:21.340104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa0d9d-a1eb-4bcb-80f1-3ab3095befb8 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.457145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2e7de-566b-40cf-9d27-4725cd35badd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.463609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.488207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa0d9d-a1eb-4bcb-80f1-3ab3095befb8 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.498474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.499480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.499769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.500089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.522989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.531095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78f774-77bd-4b65-b13f-8465732a72c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.532002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78f774-77bd-4b65-b13f-8465732a72c5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.532520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78f774-77bd-4b65-b13f-8465732a72c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.532520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78f774-77bd-4b65-b13f-8465732a72c5 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.534603 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78f774-77bd-4b65-b13f-8465732a72c5 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.536297 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78f774-77bd-4b65-b13f-8465732a72c5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.538238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 181/351] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:21.540994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564cb2-6510-40eb-828e-54ee54928982 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.540994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564cb2-6510-40eb-828e-54ee54928982 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.540994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564cb2-6510-40eb-828e-54ee54928982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.566982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-61564cb2-6510-40eb-828e-54ee54928982 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:21.566982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-61564cb2-6510-40eb-828e-54ee54928982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:21.598344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.600204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.660513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 182/352] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:21.662642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd06c9-672d-41ac-8cf5-8f742c737ed6 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.662642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd06c9-672d-41ac-8cf5-8f742c737ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.662642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd06c9-672d-41ac-8cf5-8f742c737ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.686769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-bccd06c9-672d-41ac-8cf5-8f742c737ed6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:21.688035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bccd06c9-672d-41ac-8cf5-8f742c737ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:21.731933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 171/353] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:21.731933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.774979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 183/354] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:21.775189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd06c9-672d-41ac-8cf5-8f742c737ed6 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.793291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.802696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.859110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162af2ac-61f9-4ec2-9523-6744ef9c0f4f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.896781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.897648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.897935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.898474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.899946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.901394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.901628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd06c9-672d-41ac-8cf5-8f742c737ed6 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.903108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 172/355] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:21.903889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.945038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.946442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.946727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.947005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.954503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.956012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.957695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 184/356] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:21.958460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.031321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116fe0-0332-41ed-b2c1-ec018dbb69ce None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.050164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.059064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.075251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.076082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.076991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.076991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.087085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c50c7-c91d-4909-a194-bf352b9b1c3d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.088760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:22.096845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:22.097775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 173/357] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:22.099006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.127349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5d2de-a1ae-4f47-9263-5e12aab00bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.128543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d2de-a1ae-4f47-9263-5e12aab00bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.128835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d2de-a1ae-4f47-9263-5e12aab00bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.129114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5d2de-a1ae-4f47-9263-5e12aab00bf0 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.130763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d2de-a1ae-4f47-9263-5e12aab00bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:22.133318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5d2de-a1ae-4f47-9263-5e12aab00bf0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:22.135190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 185/358] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:22.136586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952afb-5423-4e8d-9886-9ff4f922d557 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.136876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952afb-5423-4e8d-9886-9ff4f922d557 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.137160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952afb-5423-4e8d-9886-9ff4f922d557 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.138519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 186/359] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:22.139095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87952afb-5423-4e8d-9886-9ff4f922d557 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.220180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400fe03-90ed-4921-bd25-fea9f0450dd9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.276040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87952afb-5423-4e8d-9886-9ff4f922d557 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.276371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.277081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.277355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.277630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.290462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:22.299944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:22.301000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:22.301814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:22.303760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 174/360] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:22.303760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.319789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.320677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.320966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.321247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.323365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:22.324874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:22.327715 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 187/361] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:22.328530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.346295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.349408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.423179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8d4c9-c1b2-47f1-aecf-27fbe50ccd5d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.442419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c99d0-5618-4e17-8f71-a65107d0c972 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.459784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.460463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.460747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.461535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.462386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=70324050ec23474180b5bb70f04382d0) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:22.462704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:22.464182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:22.470404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 175/362] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:22.471442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.494457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe45a49-ff74-44b1-a1cd-ca2448e9865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.495153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45a49-ff74-44b1-a1cd-ca2448e9865f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.495437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45a49-ff74-44b1-a1cd-ca2448e9865f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.495713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45a49-ff74-44b1-a1cd-ca2448e9865f None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.496583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe45a49-ff74-44b1-a1cd-ca2448e9865f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:22.496900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe45a49-ff74-44b1-a1cd-ca2448e9865f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:22.498334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe45a49-ff74-44b1-a1cd-ca2448e9865f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:22.511054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 188/363] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:22.511054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a181-9bef-4afc-962f-116f84278a7a None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.511054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a181-9bef-4afc-962f-116f84278a7a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.511054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a181-9bef-4afc-962f-116f84278a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.532160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7f59a181-9bef-4afc-962f-116f84278a7a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:22.533413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7f59a181-9bef-4afc-962f-116f84278a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:22.630889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 189/364] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:22.630889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59a181-9bef-4afc-962f-116f84278a7a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.635059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.637526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec278db5-1d98-4d7f-a036-2a8a44f5b5d6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.646791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.689551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.690478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.690763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.691094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.692385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:22.693840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:22.696029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 176/365] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:22.696876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.781952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59a181-9bef-4afc-962f-116f84278a7a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.824900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.825419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3aa1a-ddb9-422d-a813-693278ec5acf None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:22.826192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.826674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.827155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.828157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:22.828682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:22.830421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:22.840422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 190/366] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:22.841811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.865681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:22.866340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:22.866625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:22.866902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:22.867785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=70324050ec23474180b5bb70f04382d0) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:22.868133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:22.869564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:22.883932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 177/367] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:22.883932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:22.920419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.948593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:22.986607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849167a-290d-4901-8f7e-58fa81708530 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.019864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a940522-8b53-4fc1-ab9d-f6d082840b27 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.028202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.029164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.029458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.030314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] PATH_INFO: `/v3/users/b8c82c515fa94669b3112352d15deb2a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.031643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] RBAC: Authorizing `identity:get_user(user_id=b8c82c515fa94669b3112352d15deb2a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.033865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.036248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 191/368] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/users/b8c82c515fa94669b3112352d15deb2a => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.037680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.064763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.065209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.065498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.065777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.078503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:23.086701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:23.087586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:23.088436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 178/369] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:23.089090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.159667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff78c1-45d4-4bae-a11e-e08266acac85 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.185296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.194321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45525d38-8f9c-4634-b3e8-80c39a9d7dd3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.206951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.207922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.208474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.208951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.210047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=49a0475e9f4540ef95098d15f52a6b28) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:23.210533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.212353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.219125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 192/370] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.220429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.224276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.231294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.231992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.232282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.232561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.307936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.309846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.347751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cb322-89df-4570-8cda-a4775aec3912 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.389328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.390388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.390867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.391337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.392435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] RBAC: Adding query filter params (role.id=55a716d562db424fbfa1add5c62abd9a, user.id=b8c82c515fa94669b3112352d15deb2a, scope.system=all) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:23.392970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.394942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.409803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 193/371] 10.222.0.62 () {56 vars in 1308 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/role_assignments?user.id=b8c82c515fa94669b3112352d15deb2a&scope.system=all&role.id=55a716d562db424fbfa1add5c62abd9a => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.411034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.417290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.430152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 179/372] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:23.436919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.486363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.534799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71569a2-287c-4f7a-a7f7-d2ed11418ca0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.572595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.573663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.574171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.574668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.575672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:23.576381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.578116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.589023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 194/373] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.597690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.604568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bbb24-1fd7-417e-803e-57f679dde858 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.640748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.641425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.641711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.641988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.643202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.645093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.651843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 180/374] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.652451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.748665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.796114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611b178-aa0d-4a6e-be51-9dcd6e52fbc8 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.803525 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f079601-97ce-4fa9-872b-c76b7d9abfb0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.825581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.846370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.847073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.847328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.847605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.907139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.908412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.909014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.909481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.915580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.917375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.919636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 181/375] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:23.924910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.925684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.926423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.037479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 195/376] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:24.051284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.094948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.111173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c4765-4c55-4b1e-b917-df0148219565 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.155948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.155948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59a233-d618-4390-abda-1469627795ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.155948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59a233-d618-4390-abda-1469627795ee None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.155948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59a233-d618-4390-abda-1469627795ee None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.156358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59a233-d618-4390-abda-1469627795ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.156834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59a233-d618-4390-abda-1469627795ee None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=49a0475e9f4540ef95098d15f52a6b28) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.157151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59a233-d618-4390-abda-1469627795ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.158582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59a233-d618-4390-abda-1469627795ee None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.174795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 182/377] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:24.230271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab383-8a6c-4e0b-ac43-9cf5af5bb15d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.280517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.281415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.281694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.281968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] PATH_INFO: `/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.283895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] RBAC: Authorizing `identity:get_project(project_id=d52b88b861a746ccb689f0f6a32f3ad1)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.285417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.290461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 196/378] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1 => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:24.295598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59a233-d618-4390-abda-1469627795ee None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.445088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59a233-d618-4390-abda-1469627795ee None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.482624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bc967-0c27-4557-9970-c287aaf0880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.483493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bc967-0c27-4557-9970-c287aaf0880f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.483788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bc967-0c27-4557-9970-c287aaf0880f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.484079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bc967-0c27-4557-9970-c287aaf0880f None admin] PATH_INFO: `/v3/groups/fe136888e0c5493fac0d6113fb58fb17` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.491486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bc967-0c27-4557-9970-c287aaf0880f None admin] RBAC: Authorizing `identity:get_group(group_id=fe136888e0c5493fac0d6113fb58fb17)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.493050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bc967-0c27-4557-9970-c287aaf0880f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.495119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 183/379] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/groups/fe136888e0c5493fac0d6113fb58fb17 => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:24.500528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.604580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.613884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a2d2b-44f2-4af3-aa3f-595c8a65f535 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.657775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b052e85-1203-47ad-84d2-3f1ab3e960ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.658437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b052e85-1203-47ad-84d2-3f1ab3e960ae None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.658720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b052e85-1203-47ad-84d2-3f1ab3e960ae None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.658999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b052e85-1203-47ad-84d2-3f1ab3e960ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.659846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b052e85-1203-47ad-84d2-3f1ab3e960ae None admin] RBAC: Adding query filter params (group.id=fe136888e0c5493fac0d6113fb58fb17, role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=d52b88b861a746ccb689f0f6a32f3ad1) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.660168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b052e85-1203-47ad-84d2-3f1ab3e960ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.661581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b052e85-1203-47ad-84d2-3f1ab3e960ae None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.671913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 197/380] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/role_assignments?group.id=fe136888e0c5493fac0d6113fb58fb17&scope.project.id=d52b88b861a746ccb689f0f6a32f3ad1&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:25.504398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef338fe5-a2a0-4775-8453-8a9010c4817d None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.504957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef338fe5-a2a0-4775-8453-8a9010c4817d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.504957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef338fe5-a2a0-4775-8453-8a9010c4817d None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.506614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 184/381] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:25.512141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0002ba-ec98-4c31-9c7d-5e7a1a2271e4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.512573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0002ba-ec98-4c31-9c7d-5e7a1a2271e4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.512764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0002ba-ec98-4c31-9c7d-5e7a1a2271e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.535731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-aa0002ba-ec98-4c31-9c7d-5e7a1a2271e4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:25.536942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-aa0002ba-ec98-4c31-9c7d-5e7a1a2271e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:25.630428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 198/382] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:25.660770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef338fe5-a2a0-4775-8453-8a9010c4817d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.746264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:25.792611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ae895-f833-40b6-b6e0-28490cb6bde9 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.792796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ae895-f833-40b6-b6e0-28490cb6bde9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.793181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ae895-f833-40b6-b6e0-28490cb6bde9 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.794579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef338fe5-a2a0-4775-8453-8a9010c4817d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.795012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 199/383] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:25.802435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158b44a-7f57-420f-abee-dfb32bee0d1b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.802435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158b44a-7f57-420f-abee-dfb32bee0d1b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.802435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158b44a-7f57-420f-abee-dfb32bee0d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.834370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.834370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.834370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.834370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.836836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b158b44a-7f57-420f-abee-dfb32bee0d1b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:25.838256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b158b44a-7f57-420f-abee-dfb32bee0d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:25.851979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:25.854538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:25.855418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:25.856270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 185/384] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:25.862666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.990028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 200/385] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:26.017196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98edea-0672-483e-b92b-2a8ddc85f6ce None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.034739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158b44a-7f57-420f-abee-dfb32bee0d1b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.043848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.062231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.062408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1403f-8b6d-4a34-9d88-821d07df27b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.063295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1403f-8b6d-4a34-9d88-821d07df27b9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.063589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1403f-8b6d-4a34-9d88-821d07df27b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.064009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1403f-8b6d-4a34-9d88-821d07df27b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.064780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1403f-8b6d-4a34-9d88-821d07df27b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:26.065095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1403f-8b6d-4a34-9d88-821d07df27b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.068405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1403f-8b6d-4a34-9d88-821d07df27b9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.076040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 186/386] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:26.077328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283f483-9473-43f4-a3f2-c33f6b6ab682 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.077618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283f483-9473-43f4-a3f2-c33f6b6ab682 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.077902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283f483-9473-43f4-a3f2-c33f6b6ab682 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.079278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 187/387] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:26.083837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283f483-9473-43f4-a3f2-c33f6b6ab682 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.170944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158b44a-7f57-420f-abee-dfb32bee0d1b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.198609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283f483-9473-43f4-a3f2-c33f6b6ab682 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.214359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.216063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.216822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.217304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.232388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:26.235763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384857d4-9712-423c-a016-da40834c300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.236688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384857d4-9712-423c-a016-da40834c300c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.237024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384857d4-9712-423c-a016-da40834c300c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.237299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384857d4-9712-423c-a016-da40834c300c None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.238531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384857d4-9712-423c-a016-da40834c300c None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.240117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384857d4-9712-423c-a016-da40834c300c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.242133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 188/388] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:26.242597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:26.243477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:26.244322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:26.244929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887132f-ca35-4a4d-9cfc-dca539907ed9 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.244929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887132f-ca35-4a4d-9cfc-dca539907ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.244929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887132f-ca35-4a4d-9cfc-dca539907ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.248433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 201/389] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:26.253230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.267060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1887132f-ca35-4a4d-9cfc-dca539907ed9 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:26.268593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1887132f-ca35-4a4d-9cfc-dca539907ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:26.316796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.347657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 189/390] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:26.348994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887132f-ca35-4a4d-9cfc-dca539907ed9 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.349775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.393197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee8030-c670-45d0-befd-efa18f1df719 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.432745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.434124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.434987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.435621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.437340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.439606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.442581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 202/391] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:26.443577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.490889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887132f-ca35-4a4d-9cfc-dca539907ed9 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.530507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.531992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.532484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.532991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.533989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:26.534498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.536122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.549698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 190/392] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:26.550982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.558416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c8722-8d81-4eb0-808c-51159efec1cc None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.597850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.598653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.598938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.599223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.599867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.610604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:26.618634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:26.619628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:26.620474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:26.624179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 203/393] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:26.624902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.628145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.672296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef82c-f838-4842-b138-3325ad88d9fc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.708508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.709725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.710260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.710770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0/groups/f932b3090f0246a4929029f855c295ec/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.713648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=70324050ec23474180b5bb70f04382d0, group_id=f932b3090f0246a4929029f855c295ec, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.715620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.733448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 191/394] 10.222.0.62 () {58 vars in 1442 bytes} [Fri May 27 03:10:26 2022] PUT /identity/v3/projects/70324050ec23474180b5bb70f04382d0/groups/f932b3090f0246a4929029f855c295ec/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:26.734694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.735509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701ebe4-c222-4c4e-975b-40f257d8c14b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.777409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034abd0d-6639-41b6-8885-b2db489091a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.778517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034abd0d-6639-41b6-8885-b2db489091a0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.778654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034abd0d-6639-41b6-8885-b2db489091a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.778957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034abd0d-6639-41b6-8885-b2db489091a0 None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.780201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034abd0d-6639-41b6-8885-b2db489091a0 None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.781761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034abd0d-6639-41b6-8885-b2db489091a0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.783562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 204/395] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:26.789584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59386015-d31c-44ca-85c7-01f4000a02a5 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.789898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59386015-d31c-44ca-85c7-01f4000a02a5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.790153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59386015-d31c-44ca-85c7-01f4000a02a5 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.791883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 205/396] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:26.792722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59386015-d31c-44ca-85c7-01f4000a02a5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:26.879772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.884740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfe4fa-3ed0-4361-8d89-614d1a847ca4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.887130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:26.934618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c9e19-4405-471b-993a-27b48d2029b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.935310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c9e19-4405-471b-993a-27b48d2029b3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.935600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c9e19-4405-471b-993a-27b48d2029b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.935966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c9e19-4405-471b-993a-27b48d2029b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.936682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c9e19-4405-471b-993a-27b48d2029b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:26.936995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c9e19-4405-471b-993a-27b48d2029b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.938700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c9e19-4405-471b-993a-27b48d2029b3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.949805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 192/397] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:26.951931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507cbef-6f7e-4bbc-b749-00b49b65352a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.952427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507cbef-6f7e-4bbc-b749-00b49b65352a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.952550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59386015-d31c-44ca-85c7-01f4000a02a5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:26.953023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507cbef-6f7e-4bbc-b749-00b49b65352a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.971437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1507cbef-6f7e-4bbc-b749-00b49b65352a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:26.972851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1507cbef-6f7e-4bbc-b749-00b49b65352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:26.988663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:26.989536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.989915 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.990196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.991384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:26.993423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:26.997561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 206/398] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:27.000539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.045684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 193/399] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:27.046510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507cbef-6f7e-4bbc-b749-00b49b65352a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.127357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911631a3-f6a3-45d1-ac99-0b3718eb6280 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.168135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.168951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.169239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.169521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.170724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.172214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.175602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.178562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 207/400] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:26 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:27.179433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.188576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.189066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507cbef-6f7e-4bbc-b749-00b49b65352a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.232236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.233128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.233705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.233705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0/groups/f932b3090f0246a4929029f855c295ec/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.236001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] RBAC: Authorizing `identity:create_grant(project_id=70324050ec23474180b5bb70f04382d0, group_id=f932b3090f0246a4929029f855c295ec, role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.237456 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.252664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 194/401] 10.222.0.62 () {58 vars in 1442 bytes} [Fri May 27 03:10:27 2022] PUT /identity/v3/projects/70324050ec23474180b5bb70f04382d0/groups/f932b3090f0246a4929029f855c295ec/roles/cabb92ea03d84bd399329e57ac452f9c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:27.253364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.338350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca17be4-b086-411d-8808-8b9ad3a4a791 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.424646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.425394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.425676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.425955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.446755 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:27.455727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:27.456837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:27.457841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:27.459995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 208/402] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:27.466030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356f63d-e8a6-463a-b07b-a41274fea36d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.471974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.507415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fc017-e689-4811-a869-3861ce3aebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.508791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fc017-e689-4811-a869-3861ce3aebf4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.509070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fc017-e689-4811-a869-3861ce3aebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.509352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fc017-e689-4811-a869-3861ce3aebf4 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.510544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fc017-e689-4811-a869-3861ce3aebf4 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.512017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fc017-e689-4811-a869-3861ce3aebf4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.514748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 195/403] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:27.515393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14973-a75f-4c0b-adee-ecacbbc52dcf None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.515736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14973-a75f-4c0b-adee-ecacbbc52dcf None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.516053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14973-a75f-4c0b-adee-ecacbbc52dcf None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.517592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 196/404] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:27.519033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14973-a75f-4c0b-adee-ecacbbc52dcf None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.552204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.556157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.597295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3201b-b425-4dcc-a86d-ef31f33812e4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.630322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14973-a75f-4c0b-adee-ecacbbc52dcf None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.636084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eba80e-be46-4347-ac57-28883c4c8316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.636774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eba80e-be46-4347-ac57-28883c4c8316 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.637059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eba80e-be46-4347-ac57-28883c4c8316 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.637339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eba80e-be46-4347-ac57-28883c4c8316 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.638107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eba80e-be46-4347-ac57-28883c4c8316 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:27.638420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eba80e-be46-4347-ac57-28883c4c8316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.639804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eba80e-be46-4347-ac57-28883c4c8316 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.648859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 209/405] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:27.650213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c806c-3de1-4444-9800-1597ba317626 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.650510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c806c-3de1-4444-9800-1597ba317626 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.650795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c806c-3de1-4444-9800-1597ba317626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.667525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-346c806c-3de1-4444-9800-1597ba317626 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:27.668729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-346c806c-3de1-4444-9800-1597ba317626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:27.673890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.674831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.675063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.675336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28/groups/f932b3090f0246a4929029f855c295ec/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.677847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] RBAC: Authorizing `identity:create_grant(project_id=49a0475e9f4540ef95098d15f52a6b28, group_id=f932b3090f0246a4929029f855c295ec, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.679293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.698490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 197/406] 10.222.0.62 () {58 vars in 1442 bytes} [Fri May 27 03:10:27 2022] PUT /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28/groups/f932b3090f0246a4929029f855c295ec/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:10:27.699529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.816195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.824160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c1d9a-4b7c-4f44-8df6-f64e4e45c489 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.828283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 210/407] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:27.862312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c806c-3de1-4444-9800-1597ba317626 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.880359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.881243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.881520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.881792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.887197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.891206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.893062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 198/408] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:27.899728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.988076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.048791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6286fc-6136-4033-ad70-1ac46df0c4db None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.081941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c806c-3de1-4444-9800-1597ba317626 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.086678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.087542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.087841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.088127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.089329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.091034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.093310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 199/409] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:28.104923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.127813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.134464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.135279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.135570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.135898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.157106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:28.173077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:28.174011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:28.174854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 211/410] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:28.180401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.222816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25f860-428f-481a-b601-d6785d1813bc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.292050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.292944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.293225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.293501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28/groups/f932b3090f0246a4929029f855c295ec/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.301989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] RBAC: Authorizing `identity:create_grant(project_id=49a0475e9f4540ef95098d15f52a6b28, group_id=f932b3090f0246a4929029f855c295ec, role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.303447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.326982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 200/411] 10.222.0.62 () {58 vars in 1442 bytes} [Fri May 27 03:10:28 2022] PUT /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28/groups/f932b3090f0246a4929029f855c295ec/roles/cabb92ea03d84bd399329e57ac452f9c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:28.339068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa6713-1a2a-4213-b345-0d1033e416ab None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.361590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.378018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.378710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.378996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.379274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.380085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:28.380398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.381968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.392094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 212/412] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:28.401019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.511245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.543352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12610b-14ae-426d-9830-354270543d42 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.581698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.582575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.582859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.583144 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] PATH_INFO: `/v3/groups/fe136888e0c5493fac0d6113fb58fb17` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.584426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] RBAC: Authorizing `identity:get_group(group_id=fe136888e0c5493fac0d6113fb58fb17)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.586009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.587957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 201/413] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/groups/fe136888e0c5493fac0d6113fb58fb17 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:28.593640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.703389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93303ee-03c4-4d30-9155-f1b92f46826d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.739621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7784b1e-7fde-4b82-81d1-2c9c5d632182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.740524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784b1e-7fde-4b82-81d1-2c9c5d632182 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.740809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784b1e-7fde-4b82-81d1-2c9c5d632182 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.741086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784b1e-7fde-4b82-81d1-2c9c5d632182 None admin] PATH_INFO: `/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.742308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7784b1e-7fde-4b82-81d1-2c9c5d632182 None admin] RBAC: Authorizing `identity:get_project(project_id=d52b88b861a746ccb689f0f6a32f3ad1)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.743753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7784b1e-7fde-4b82-81d1-2c9c5d632182 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.745776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 213/414] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:28.751309 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.880862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9c164-645c-45d4-8dec-4fe8ad3d860c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.929711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec43497-7f1a-402c-ae7e-0b6c4342066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.930602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43497-7f1a-402c-ae7e-0b6c4342066e None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.930887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43497-7f1a-402c-ae7e-0b6c4342066e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.931170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43497-7f1a-402c-ae7e-0b6c4342066e None admin] PATH_INFO: `/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1/groups/fe136888e0c5493fac0d6113fb58fb17/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.933243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec43497-7f1a-402c-ae7e-0b6c4342066e None admin] RBAC: Authorizing `identity:create_grant(project_id=d52b88b861a746ccb689f0f6a32f3ad1, group_id=fe136888e0c5493fac0d6113fb58fb17, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.934704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec43497-7f1a-402c-ae7e-0b6c4342066e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.950429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 202/415] 10.222.0.62 () {58 vars in 1442 bytes} [Fri May 27 03:10:28 2022] PUT /identity/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1/groups/fe136888e0c5493fac0d6113fb58fb17/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:29.634445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f161-e0f6-4a54-84c5-a295dc010f5a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.634732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f161-e0f6-4a54-84c5-a295dc010f5a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.635170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f161-e0f6-4a54-84c5-a295dc010f5a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.637030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 214/416] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:29.642338 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6226-0a23-4d6a-9ae2-4e3fefac0856 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.643129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6226-0a23-4d6a-9ae2-4e3fefac0856 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.643563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6226-0a23-4d6a-9ae2-4e3fefac0856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.649447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:29.663581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2b5a6226-0a23-4d6a-9ae2-4e3fefac0856 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:29.665203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2b5a6226-0a23-4d6a-9ae2-4e3fefac0856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:29.742972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 203/417] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:29.776827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f161-e0f6-4a54-84c5-a295dc010f5a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.849472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:29.916639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f161-e0f6-4a54-84c5-a295dc010f5a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:29.956421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c9a54-4659-4edc-8a25-ab75bfd0f884 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.956960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c9a54-4659-4edc-8a25-ab75bfd0f884 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.957497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c9a54-4659-4edc-8a25-ab75bfd0f884 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.963349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 204/418] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:29.964013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:29.964834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.965122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.965654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.967964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd59da-4320-471e-ae6f-e09e7dc6cb9d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.968453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd59da-4320-471e-ae6f-e09e7dc6cb9d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.968947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd59da-4320-471e-ae6f-e09e7dc6cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.988864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:29.999123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:30.000183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:30.000977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:30.001590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 215/419] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:30.006142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-66cd59da-4320-471e-ae6f-e09e7dc6cb9d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:30.006769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.008326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-66cd59da-4320-471e-ae6f-e09e7dc6cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:30.111624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace25eae-3f92-4446-a752-6c98ff596069 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.122080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.145271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 205/420] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:30.155220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.156066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.156439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.156812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.157672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:30.158085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.159606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.168996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 216/421] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:30.176966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd59da-4320-471e-ae6f-e09e7dc6cb9d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.181628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.320800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.338340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd59da-4320-471e-ae6f-e09e7dc6cb9d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.386227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f6a38-0e51-4c45-88d3-9566d798fc64 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.404930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd5441-73d5-4393-a735-9562df5afda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.405589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5441-73d5-4393-a735-9562df5afda8 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.405866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5441-73d5-4393-a735-9562df5afda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.406140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5441-73d5-4393-a735-9562df5afda8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.451169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422e1e7-b72a-4c80-aa90-27b0799b0502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.451920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422e1e7-b72a-4c80-aa90-27b0799b0502 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.452206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422e1e7-b72a-4c80-aa90-27b0799b0502 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.452480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422e1e7-b72a-4c80-aa90-27b0799b0502 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.469128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422e1e7-b72a-4c80-aa90-27b0799b0502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-7422e1e7-b72a-4c80-aa90-27b0799b0502 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:30.477255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:30.478218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:30.479003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 217/422] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:30.484252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fa408-4170-4799-8917-8f2110b5a9b8 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.484252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fa408-4170-4799-8917-8f2110b5a9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.484252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fa408-4170-4799-8917-8f2110b5a9b8 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.484252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 218/423] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:30.487354 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fa408-4170-4799-8917-8f2110b5a9b8 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.515262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd5441-73d5-4393-a735-9562df5afda8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.517223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd5441-73d5-4393-a735-9562df5afda8 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.520515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.587712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.601902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fa408-4170-4799-8917-8f2110b5a9b8 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.640108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.640108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.640108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.640108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.640568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:30.640791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.641138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 206/424] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:30.642357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.643000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cacbd-0f3d-4071-bdd1-8635a6e5a777 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.643790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cacbd-0f3d-4071-bdd1-8635a6e5a777 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.644236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cacbd-0f3d-4071-bdd1-8635a6e5a777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.653507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 219/425] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:30.654566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.663161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d42cacbd-0f3d-4071-bdd1-8635a6e5a777 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:30.664781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d42cacbd-0f3d-4071-bdd1-8635a6e5a777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:30.737599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 207/426] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:30.738817 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cacbd-0f3d-4071-bdd1-8635a6e5a777 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.772199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.782789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565cd6-eec7-418d-9e95-46e9be8ec26a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.827843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.827843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.827843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.827843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.828527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.830060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.832199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 220/427] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:30.833055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.837091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.854374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cacbd-0f3d-4071-bdd1-8635a6e5a777 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.892352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fe75f-1c2c-4238-b8f8-a62d9f3834e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.893178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe75f-1c2c-4238-b8f8-a62d9f3834e5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.893810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe75f-1c2c-4238-b8f8-a62d9f3834e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.894095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe75f-1c2c-4238-b8f8-a62d9f3834e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.947721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970778c-e6eb-4f6e-ab52-9d005431bce1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.970980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fe75f-1c2c-4238-b8f8-a62d9f3834e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.972472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fe75f-1c2c-4238-b8f8-a62d9f3834e5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.984519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.985319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.985605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.985881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.997619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:31.005252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:31.006145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:31.006969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 221/428] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:31.007661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.033627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.067750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.082301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 208/429] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:31.082301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f87d-d229-48ef-9729-b1fe57529837 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.082301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f87d-d229-48ef-9729-b1fe57529837 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.082301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f87d-d229-48ef-9729-b1fe57529837 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.085295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 209/430] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:31.086090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519f87d-d229-48ef-9729-b1fe57529837 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.120344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ec41b-e6ad-49b1-8747-75da76b85f3b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.160900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.162607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.162896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.163176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.164913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.166866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.173740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 222/431] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.173740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.193626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519f87d-d229-48ef-9729-b1fe57529837 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.229765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113afee-ebe0-4119-a7f6-91bf43763f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.230435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113afee-ebe0-4119-a7f6-91bf43763f04 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.230719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113afee-ebe0-4119-a7f6-91bf43763f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.230997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113afee-ebe0-4119-a7f6-91bf43763f04 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.231753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113afee-ebe0-4119-a7f6-91bf43763f04 None admin] RBAC: Adding query filter params (name=member) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:31.232095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113afee-ebe0-4119-a7f6-91bf43763f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.233486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113afee-ebe0-4119-a7f6-91bf43763f04 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.242638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 210/432] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.243688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712b563-e216-48be-a688-adf0915bf61a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.244008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712b563-e216-48be-a688-adf0915bf61a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.244293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712b563-e216-48be-a688-adf0915bf61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.261022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-6712b563-e216-48be-a688-adf0915bf61a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:31.262121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6712b563-e216-48be-a688-adf0915bf61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:31.282567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.297342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2879468-b9a6-4181-9ac7-107f060056ff None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.314869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.333184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.343993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.344117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 211/433] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:31.344202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.344202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] PATH_INFO: `/v3/projects/70324050ec23474180b5bb70f04382d0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.344352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712b563-e216-48be-a688-adf0915bf61a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.345238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] RBAC: Authorizing `identity:get_project(project_id=70324050ec23474180b5bb70f04382d0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.346885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.348957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 223/434] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/projects/70324050ec23474180b5bb70f04382d0 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.349800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.453964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712b563-e216-48be-a688-adf0915bf61a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.464324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22c33a-6db6-4cec-96f1-eaffc9b0bf14 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.488878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.489551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.489831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.490104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.490976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=70324050ec23474180b5bb70f04382d0) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:31.491287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.492867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.499497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 212/435] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.500398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.530663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.531264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.531546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.531841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.568452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.571278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.625055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.626873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.627445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb9f8-711c-4738-8535-f7691b178dd5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.666284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.667156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.667437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.667711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.669011 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.670607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.676690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 213/436] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.676690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.753495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 224/437] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:31.754085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.793716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff9a5a-4f93-4d69-87c5-725c4502544d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.812441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.824926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.836552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5aee7-fc25-4ac0-bf84-fadd80c478e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.837369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5aee7-fc25-4ac0-bf84-fadd80c478e1 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.837646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5aee7-fc25-4ac0-bf84-fadd80c478e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.837915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5aee7-fc25-4ac0-bf84-fadd80c478e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.849864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5aee7-fc25-4ac0-bf84-fadd80c478e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-57a5aee7-fc25-4ac0-bf84-fadd80c478e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:31.858007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:31.858889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:31.860208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 214/438] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:31.861398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdf8ee-4c18-49a5-9f43-9d531d699a12 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.861680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdf8ee-4c18-49a5-9f43-9d531d699a12 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.861955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdf8ee-4c18-49a5-9f43-9d531d699a12 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.863209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 215/439] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:31.863928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdf8ee-4c18-49a5-9f43-9d531d699a12 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.885459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7fba1-413f-491c-a802-32675fde9d4f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.926554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.927193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.927480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.927757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.928639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=70324050ec23474180b5bb70f04382d0) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:31.928951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.930438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.936599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 225/440] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=70324050ec23474180b5bb70f04382d0&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.937375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.985934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdf8ee-4c18-49a5-9f43-9d531d699a12 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.024962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4cbb8-3130-4c39-b67e-7461ca18efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.026019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4cbb8-3130-4c39-b67e-7461ca18efb1 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.026434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4cbb8-3130-4c39-b67e-7461ca18efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.026747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4cbb8-3130-4c39-b67e-7461ca18efb1 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.028191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4cbb8-3130-4c39-b67e-7461ca18efb1 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.029947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4cbb8-3130-4c39-b67e-7461ca18efb1 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.036391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 216/441] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:32.036391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae81c1-f26e-4919-9cc5-1735ce5b1663 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.036391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae81c1-f26e-4919-9cc5-1735ce5b1663 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.036391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae81c1-f26e-4919-9cc5-1735ce5b1663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.050886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e9966-528a-4afe-b9a1-0f1cd5b16f33 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.052877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c2ae81c1-f26e-4919-9cc5-1735ce5b1663 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:32.054749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c2ae81c1-f26e-4919-9cc5-1735ce5b1663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:32.073222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.091435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.092160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.092455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.092737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.093517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:32.093833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.095245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.108824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 226/442] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:32.108824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.109134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.146826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 217/443] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:32.152621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae81c1-f26e-4919-9cc5-1735ce5b1663 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.223919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7e98f-8beb-4922-8836-e988bedb8071 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.263258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.264164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.264452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.264736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.265943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.267395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.269867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 227/444] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:32.270531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.273818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae81c1-f26e-4919-9cc5-1735ce5b1663 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.314904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.315481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.315771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.316088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.332460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.363762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.387304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8199a-bd25-4de7-807d-1931eef588a3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.389502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.390970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.420736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.421528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.421816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.422137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.432674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:32.440347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:32.441296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:32.442095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 228/445] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:32.442758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.493874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 218/446] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:32.494706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.543908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca05fc-4a03-4501-8e5e-3320a05d99a6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.553759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.566540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.578316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.578962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.579252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.579531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.580465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=0ca4c2474a5147bbb6bcd63248ea8ef8, scope.project.id=49a0475e9f4540ef95098d15f52a6b28) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:32.580787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.582225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.599852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 229/447] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:32.599852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.600122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d58491-323b-4039-892a-d4942fa67397 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.633671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.634354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.634750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.635028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.635785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:32.636234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.637622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.652097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 219/448] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:32.653246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.699650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3757b-bf74-4e04-8388-e6928f40a33c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.744515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.745409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.745686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.745967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] PATH_INFO: `/v3/projects/49a0475e9f4540ef95098d15f52a6b28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.747232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] RBAC: Authorizing `identity:get_project(project_id=49a0475e9f4540ef95098d15f52a6b28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.748823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.750805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 230/449] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/projects/49a0475e9f4540ef95098d15f52a6b28 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:32.756242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.766543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef3564-a09d-4edd-aa1e-ce358ca2d21a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.808574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.809223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.809506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.809748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.809861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.816350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:32.882101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592f4fe-b9a9-4af9-a3e3-796ec2d0e563 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:32.895461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.896981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.924538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:32.925406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.925695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.925974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] PATH_INFO: `/v3/groups/f932b3090f0246a4929029f855c295ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.927204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] RBAC: Authorizing `identity:get_group(group_id=f932b3090f0246a4929029f855c295ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:32.928717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:32.930448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 231/450] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/groups/f932b3090f0246a4929029f855c295ec => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:32.936263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.016816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 220/451] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:33.022396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.042706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422764-3a8e-4fc5-b660-089f5538ce76 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.053051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.055586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.076645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.077295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.077587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.077866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.078701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] RBAC: Adding query filter params (group.id=f932b3090f0246a4929029f855c295ec, role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=49a0475e9f4540ef95098d15f52a6b28) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:33.079013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.080490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.086674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 232/452] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/role_assignments?group.id=f932b3090f0246a4929029f855c295ec&scope.project.id=49a0475e9f4540ef95098d15f52a6b28&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:33.120426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbba22-df14-450b-9246-5914de8dd56c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.156230 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.157018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.157299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.157571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] PATH_INFO: `/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.158778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] RBAC: Authorizing `identity:get_project(project_id=d52b88b861a746ccb689f0f6a32f3ad1)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.160278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.162044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 221/453] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/projects/d52b88b861a746ccb689f0f6a32f3ad1 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:33.168405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.275526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99951f92-bbc7-43b7-a41f-4b46d056e25c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.310747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d2f48-b529-4749-8943-ef2aa3efb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.311601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2f48-b529-4749-8943-ef2aa3efb229 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.311920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2f48-b529-4749-8943-ef2aa3efb229 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.312204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2f48-b529-4749-8943-ef2aa3efb229 None admin] PATH_INFO: `/v3/groups/fe136888e0c5493fac0d6113fb58fb17` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.313384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d2f48-b529-4749-8943-ef2aa3efb229 None admin] RBAC: Authorizing `identity:get_group(group_id=fe136888e0c5493fac0d6113fb58fb17)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.314819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d2f48-b529-4749-8943-ef2aa3efb229 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.316480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 233/454] 10.222.0.62 () {56 vars in 1180 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/groups/fe136888e0c5493fac0d6113fb58fb17 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:33.322846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.423033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83dd7b-8824-4786-90e7-3c83cdf86b2d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.425345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.457670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbde687-08e2-4c6f-a10b-158ee81630eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.458324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde687-08e2-4c6f-a10b-158ee81630eb None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.458613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde687-08e2-4c6f-a10b-158ee81630eb None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.458890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde687-08e2-4c6f-a10b-158ee81630eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.459733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbde687-08e2-4c6f-a10b-158ee81630eb None admin] RBAC: Adding query filter params (group.id=fe136888e0c5493fac0d6113fb58fb17, role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=d52b88b861a746ccb689f0f6a32f3ad1) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:33.460079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbde687-08e2-4c6f-a10b-158ee81630eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.462077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbde687-08e2-4c6f-a10b-158ee81630eb None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.468294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 222/455] 10.222.0.62 () {56 vars in 1376 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/role_assignments?group.id=fe136888e0c5493fac0d6113fb58fb17&scope.project.id=d52b88b861a746ccb689f0f6a32f3ad1&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:36.084614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6bde-dbd3-4bc2-a197-ab7d68df0c0b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.086184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6bde-dbd3-4bc2-a197-ab7d68df0c0b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.086694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6bde-dbd3-4bc2-a197-ab7d68df0c0b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.088606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 234/456] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:36.095502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fed7b3-89d7-4cd7-9c51-6e3aaca842fc None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.096991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fed7b3-89d7-4cd7-9c51-6e3aaca842fc None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.097835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fed7b3-89d7-4cd7-9c51-6e3aaca842fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.128843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-14fed7b3-89d7-4cd7-9c51-6e3aaca842fc None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:36.130395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-14fed7b3-89d7-4cd7-9c51-6e3aaca842fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:36.213347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 223/457] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:36.253511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6bde-dbd3-4bc2-a197-ab7d68df0c0b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.325255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:36.408060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6bde-dbd3-4bc2-a197-ab7d68df0c0b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.465476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c0c58-bb2a-4f92-aa55-1729038c322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:36.466609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c0c58-bb2a-4f92-aa55-1729038c322b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.467099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c0c58-bb2a-4f92-aa55-1729038c322b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.467597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c0c58-bb2a-4f92-aa55-1729038c322b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.470195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab6c1-fe1d-4da7-8a06-7f27013726d6 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.470730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab6c1-fe1d-4da7-8a06-7f27013726d6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.471210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab6c1-fe1d-4da7-8a06-7f27013726d6 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.473060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 224/458] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:36.477822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1662fd-757f-4d3a-9d81-78a11b0f30e3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.478346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1662fd-757f-4d3a-9d81-78a11b0f30e3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.478818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1662fd-757f-4d3a-9d81-78a11b0f30e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.479291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c0c58-bb2a-4f92-aa55-1729038c322b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:36.483841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c0c58-bb2a-4f92-aa55-1729038c322b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-b55c0c58-bb2a-4f92-aa55-1729038c322b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:36.499871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:36.500793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:36.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:36.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:36.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:36.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:36.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 235/459] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:36.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620641f4-875e-42a6-9d40-76627ceb6941 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620641f4-875e-42a6-9d40-76627ceb6941 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.502141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620641f4-875e-42a6-9d40-76627ceb6941 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.503273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 236/460] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:36.508571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-cc1662fd-757f-4d3a-9d81-78a11b0f30e3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:36.508780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a565f-21fb-4dda-92c1-4926662232c0 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.509075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a565f-21fb-4dda-92c1-4926662232c0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.509361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a565f-21fb-4dda-92c1-4926662232c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.510518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cc1662fd-757f-4d3a-9d81-78a11b0f30e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:36.528363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b72a565f-21fb-4dda-92c1-4926662232c0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:36.529542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b72a565f-21fb-4dda-92c1-4926662232c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:36.591920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 225/461] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:36.593154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1662fd-757f-4d3a-9d81-78a11b0f30e3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.604907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 237/462] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:36.625313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029445-afa0-41e2-801f-6abbe24bac61 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.625603 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029445-afa0-41e2-801f-6abbe24bac61 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.626061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029445-afa0-41e2-801f-6abbe24bac61 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.628179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 238/463] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:36.629884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029445-afa0-41e2-801f-6abbe24bac61 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.637594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:36.659341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:36.714586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1662fd-757f-4d3a-9d81-78a11b0f30e3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.745120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029445-afa0-41e2-801f-6abbe24bac61 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.756686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd315f-a5a2-482c-a663-11c3c8324478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:36.757997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd315f-a5a2-482c-a663-11c3c8324478 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.758283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd315f-a5a2-482c-a663-11c3c8324478 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.758757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd315f-a5a2-482c-a663-11c3c8324478 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.759496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd315f-a5a2-482c-a663-11c3c8324478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:36.759810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd315f-a5a2-482c-a663-11c3c8324478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:36.761225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd315f-a5a2-482c-a663-11c3c8324478 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:36.776935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 226/464] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:36.776935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cc40-fc02-4613-9346-6cbab95a78c4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.776935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cc40-fc02-4613-9346-6cbab95a78c4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.776935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cc40-fc02-4613-9346-6cbab95a78c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.786146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:36.786941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.787284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.787981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.793038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-72d1cc40-fc02-4613-9346-6cbab95a78c4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:36.794242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-72d1cc40-fc02-4613-9346-6cbab95a78c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:36.798548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:36.800131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:36.819311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:36.820938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:36.821806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:36.821806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:36.821806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:36.821806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:36.821806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 239/465] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:36.822088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.864869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 227/466] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:36.865596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1cc40-fc02-4613-9346-6cbab95a78c4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.891363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:36.923071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:36.948325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd243-992b-4aaa-9ac1-da8d9b3561a4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.982657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1cc40-fc02-4613-9346-6cbab95a78c4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.985472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:36.986257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.986543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.986821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.997956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:36.999350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:37.016917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:37.017814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:37.018665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:37.018665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:37.018665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:37.018665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:37.018888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.018974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 240/467] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:37.019041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.019041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.019167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.019857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.021304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.023495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.055854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 228/468] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:36 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:37.055854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.157801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e06eb-fb8d-408a-8666-040e9fe926a7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.165974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.193734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.204194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.205028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.205413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.205789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.206685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:37.206814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db3b84-13c1-48b9-b86c-e81d1365d397 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.207196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.208855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.223389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 241/469] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:37.224495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.256801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.257993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.258276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.258548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.272807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.274821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:37.297111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:37.298038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:37.298899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:37.298899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:37.298899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:37.298899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:37.298899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 229/470] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:37.299679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.357262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f725b-c89c-48b7-8b7c-f7d6facc7f02 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.406098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.406772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.407056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.407856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.408607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:37.409214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.411254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.419724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a0651-133a-48b9-a6bc-581efbe42bce None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.427570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 242/471] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:37.428524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.447309 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.458447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.459221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.459575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.459964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.460792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.462392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.489218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 230/472] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:37 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:37.489891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.492013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.602910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cbe0d-bc17-468b-ab6c-3f281482883a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.681673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.683406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.683996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.684412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.686273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:37.687924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.691218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.699548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543dbe6-63bc-4e8d-af23-befd8011a79c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.715957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 243/473] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:37.720796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.766486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075d509-78b9-43d8-b51b-56bc2abec15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.767296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075d509-78b9-43d8-b51b-56bc2abec15b None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.767475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075d509-78b9-43d8-b51b-56bc2abec15b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.767725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075d509-78b9-43d8-b51b-56bc2abec15b None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.768583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075d509-78b9-43d8-b51b-56bc2abec15b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.770041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075d509-78b9-43d8-b51b-56bc2abec15b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.797592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 231/474] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:37 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:37.797592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d468-7942-482f-8569-5aa357a37527 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.797592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d468-7942-482f-8569-5aa357a37527 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.797592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d468-7942-482f-8569-5aa357a37527 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.799028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 232/475] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:37.805502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054f2fe-eb5f-4820-998e-ec66fbb1b487 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.805831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054f2fe-eb5f-4820-998e-ec66fbb1b487 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.806093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054f2fe-eb5f-4820-998e-ec66fbb1b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.833208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e054f2fe-eb5f-4820-998e-ec66fbb1b487 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:37.834769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e054f2fe-eb5f-4820-998e-ec66fbb1b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:37.855588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48409f7b-cc5a-4bee-af66-79dd967bf338 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.876238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.880900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.902795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.905056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.905341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.905726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.906538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.908076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.939813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 244/476] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:37 2022] POST /identity/v3/users => generated 249 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:37.940193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 233/477] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:37.964591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:38.091168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66464e13-aa3a-4290-8920-cc15ebf63131 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:38.146713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:38.148158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:38.148650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:38.149086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:38.150324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:38.152627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:38.165214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.sql.core [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:10:38.165214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:10:38.165214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [parameters: {'id': 'fb22cf2c233847bbb651b955040e237f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:10:38.165214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80273) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application conn.query(q) May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result.read() May 27 03:10:38.189038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return ref.to_dict() May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:10:38.190037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application next(self.gen) May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application yield resource May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application next(self.gen) May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise self.value May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application session.commit() May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:10:38.190972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.session.flush() May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application compat.raise_( May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:10:38.191878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:10:38.192756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application conn.query(q) May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result.read() May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:10:38.193578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [parameters: {'id': 'fb22cf2c233847bbb651b955040e237f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:38.194381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:10:38.197405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:38.199444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 245/478] 10.222.0.62 () {60 vars in 1131 bytes} [Fri May 27 03:10:37 2022] POST /identity/v3/roles => generated 144 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:38.199575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054f2fe-eb5f-4820-998e-ec66fbb1b487 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:38.311192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054f2fe-eb5f-4820-998e-ec66fbb1b487 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:38.348238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582df287-7c07-42c4-ac08-211cdab79120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:38.349452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df287-7c07-42c4-ac08-211cdab79120 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:38.349937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df287-7c07-42c4-ac08-211cdab79120 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:38.350454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df287-7c07-42c4-ac08-211cdab79120 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:38.362183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582df287-7c07-42c4-ac08-211cdab79120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-582df287-7c07-42c4-ac08-211cdab79120 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:38.370741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:38.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:10:38.372574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:38.375266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 234/479] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:10:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:38.380617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:38.418152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:38.507874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ab515-5e2f-47cd-a08d-29cafc2a5c03 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:38.559661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f8639-84ee-43e5-92c9-c517f55e1123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:38.560424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8639-84ee-43e5-92c9-c517f55e1123 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:38.560715 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8639-84ee-43e5-92c9-c517f55e1123 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:38.560995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8639-84ee-43e5-92c9-c517f55e1123 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:38.561766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8639-84ee-43e5-92c9-c517f55e1123 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:38.562079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8639-84ee-43e5-92c9-c517f55e1123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:38.563460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8639-84ee-43e5-92c9-c517f55e1123 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:38.573039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 246/480] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:39.707268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb873773-524d-406f-9126-231032c6d748 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:39.708361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb873773-524d-406f-9126-231032c6d748 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:39.708871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb873773-524d-406f-9126-231032c6d748 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:39.711728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 235/481] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:39.716925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022538f8-54b1-4b56-84a6-d640e0c64f59 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:39.717769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022538f8-54b1-4b56-84a6-d640e0c64f59 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:39.718278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022538f8-54b1-4b56-84a6-d640e0c64f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:39.748830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-022538f8-54b1-4b56-84a6-d640e0c64f59 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:39.750399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-022538f8-54b1-4b56-84a6-d640e0c64f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:39.834160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 247/482] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:39.873087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb873773-524d-406f-9126-231032c6d748 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:39.912151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.032695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb873773-524d-406f-9126-231032c6d748 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.079738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.080574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.080861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.081138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.096675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:40.105017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:40.105936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:40.106735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 236/483] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:40.112067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022538f8-54b1-4b56-84a6-d640e0c64f59 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.134346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.233286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022538f8-54b1-4b56-84a6-d640e0c64f59 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.276862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.277780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.278246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.278670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.279582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:40.280060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.281584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.291311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 248/484] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:40.296538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.430848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.447629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc83ec-50ea-4970-9929-b73f0fddd294 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.496702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfba39-a9d2-4078-92ce-4bf2bb3b9ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.497503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfba39-a9d2-4078-92ce-4bf2bb3b9ebf None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.497789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfba39-a9d2-4078-92ce-4bf2bb3b9ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.498070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfba39-a9d2-4078-92ce-4bf2bb3b9ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.513286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfba39-a9d2-4078-92ce-4bf2bb3b9ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.514758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfba39-a9d2-4078-92ce-4bf2bb3b9ebf None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-a0bfba39-a9d2-4078-92ce-4bf2bb3b9ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:40.538833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:40.539784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:40.540680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:40.540680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:40.540680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:40.540680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:40.540680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 237/485] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:40.546284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.675024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd59bec-b71a-4037-8327-5590f14357e9 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.675363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd59bec-b71a-4037-8327-5590f14357e9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.675600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd59bec-b71a-4037-8327-5590f14357e9 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.677143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 238/486] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:40.682269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f3cd-35e2-4b1b-b2f6-9d846bbe5717 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.682564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f3cd-35e2-4b1b-b2f6-9d846bbe5717 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.682848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f3cd-35e2-4b1b-b2f6-9d846bbe5717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.703272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-5f58f3cd-35e2-4b1b-b2f6-9d846bbe5717 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:40.704653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5f58f3cd-35e2-4b1b-b2f6-9d846bbe5717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:40.715242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.741010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc4ec-e95d-4410-a4ea-f13f2e4b2a15 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.791787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 239/487] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:40.792352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39361f90-3abc-4e25-b7a5-d2c366f5a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.792533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39361f90-3abc-4e25-b7a5-d2c366f5a10c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.792804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39361f90-3abc-4e25-b7a5-d2c366f5a10c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.799900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39361f90-3abc-4e25-b7a5-d2c366f5a10c None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.799900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39361f90-3abc-4e25-b7a5-d2c366f5a10c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:40.799900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39361f90-3abc-4e25-b7a5-d2c366f5a10c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.799900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39361f90-3abc-4e25-b7a5-d2c366f5a10c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.801677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fe0bb-b09b-4c8b-8394-cb26f4fc81b9 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.802151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fe0bb-b09b-4c8b-8394-cb26f4fc81b9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.802791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fe0bb-b09b-4c8b-8394-cb26f4fc81b9 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.804788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 240/488] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:40.805943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc099b0-ac87-45e6-8877-823cd713999a None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.806235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc099b0-ac87-45e6-8877-823cd713999a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.806510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc099b0-ac87-45e6-8877-823cd713999a None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.807859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 241/489] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:40.813255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec620b-23c9-4bc6-a38e-752ac4bc1f73 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.813255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec620b-23c9-4bc6-a38e-752ac4bc1f73 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.813255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec620b-23c9-4bc6-a38e-752ac4bc1f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.813521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 249/490] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:40.815422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34749-dd4b-4aa5-b055-2c50325dac54 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.815718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34749-dd4b-4aa5-b055-2c50325dac54 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.816052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34749-dd4b-4aa5-b055-2c50325dac54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.834018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ddf34749-dd4b-4aa5-b055-2c50325dac54 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:40.835812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ddf34749-dd4b-4aa5-b055-2c50325dac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:40.840136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-94ec620b-23c9-4bc6-a38e-752ac4bc1f73 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:40.841273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-94ec620b-23c9-4bc6-a38e-752ac4bc1f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:40.906786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.910782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 250/491] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:40.915971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf34749-dd4b-4aa5-b055-2c50325dac54 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.916055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 242/492] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:40.916055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec620b-23c9-4bc6-a38e-752ac4bc1f73 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.007376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.027116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec620b-23c9-4bc6-a38e-752ac4bc1f73 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.038677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf34749-dd4b-4aa5-b055-2c50325dac54 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.063234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.064260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.064541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.064814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.074972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.075768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.076052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:41.076154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.076350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:41.083619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:41.084565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:41.085459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:41.086063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.086132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 243/493] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:41.086132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.087265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:41.106490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:41.107435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:41.108391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 251/494] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:41.108984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.151884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.196439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457f5e1-9f91-4d2e-9a54-08333bcb2c44 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.216839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc517f-c936-4c77-9022-5184b6621d3a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.231074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.231893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.232176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.232448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.243663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:41.248069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:41.252383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:41.253264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:41.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 244/495] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:41.255406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.260241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.261060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.261349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.261627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.273528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:41.281308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:41.282215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:41.283022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 252/496] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:41.283678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.367119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c2684-d4be-4e89-bc83-a0300e597a0e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.388521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.391756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1117c-3196-43e0-b2b7-3477de5eb841 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.404397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.405059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.405339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.405621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.406614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:41.406614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.408035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.421631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 245/497] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:41.421631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.435513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.436388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.436752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.437096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.438047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:41.438439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.440045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.478998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 253/498] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:41.481278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.534101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.537653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219917c4-6033-447f-a766-7c0e7c0c0c7e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.572636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.573306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.573590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.574052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.574616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:41.574924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.576340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.584566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 246/499] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:41.585401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.594557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a231af1-0800-4d1a-9fc2-808ffecfff98 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.631682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.632335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.632617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.632893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.633652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:41.633929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.634042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.635388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.643212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 254/500] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:41.644028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.688206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ab84-be1b-4021-b01a-5f122bfd5aae None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.720434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.721220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.721503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.721777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.732410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.733841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.741177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826b12-0aaf-408f-b8dc-6285e46bce01 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:41.750732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:41.751585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:41.752507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:41.752507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:41.752507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:41.752507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:41.752507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 247/501] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:41.752773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.758489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.779412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.780083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.780368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.780649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.845474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.846313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.847736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.853887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2bb24-cf8c-4087-a805-6ff38bd9dbe0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.886469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8151bf0-9f5a-4e59-becf-14d112e254a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.887185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8151bf0-9f5a-4e59-becf-14d112e254a6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.887464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8151bf0-9f5a-4e59-becf-14d112e254a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.887737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8151bf0-9f5a-4e59-becf-14d112e254a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.898534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8151bf0-9f5a-4e59-becf-14d112e254a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.900009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8151bf0-9f5a-4e59-becf-14d112e254a6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-f8151bf0-9f5a-4e59-becf-14d112e254a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:41.917247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:41.918111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:41.918981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:41.918981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:41.918981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:41.918981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:41.918981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 248/502] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:41.919683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0366-9433-42f8-9464-72a237e121d0 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.919958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0366-9433-42f8-9464-72a237e121d0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.920235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0366-9433-42f8-9464-72a237e121d0 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.921484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 249/503] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:41.922166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d0366-9433-42f8-9464-72a237e121d0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.948593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 255/504] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:41.949436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.965374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.027210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d0366-9433-42f8-9464-72a237e121d0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.048200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ea6c-32f4-41d4-a021-98a173330a64 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.060668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.061460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.061744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.062022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.072649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.074088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.080206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e64340-e8a2-4d09-8bb6-cf77a63d6194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.080852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e64340-e8a2-4d09-8bb6-cf77a63d6194 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.081133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e64340-e8a2-4d09-8bb6-cf77a63d6194 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.081408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e64340-e8a2-4d09-8bb6-cf77a63d6194 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.081951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.082129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e64340-e8a2-4d09-8bb6-cf77a63d6194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:42.082435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e64340-e8a2-4d09-8bb6-cf77a63d6194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.083858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e64340-e8a2-4d09-8bb6-cf77a63d6194 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:42.092695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:42.093534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:42.094381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:42.094381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:42.094381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:42.094381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:42.094381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 250/505] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:42.095050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.095629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 256/506] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:42.096900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3c61-e87f-4beb-804d-46ca5c138d57 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.097201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3c61-e87f-4beb-804d-46ca5c138d57 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.097461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3c61-e87f-4beb-804d-46ca5c138d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.115269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-51df3c61-e87f-4beb-804d-46ca5c138d57 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:42.116693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-51df3c61-e87f-4beb-804d-46ca5c138d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:42.181521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 257/507] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:42.182668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3c61-e87f-4beb-804d-46ca5c138d57 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.184822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.210895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59877fa-bcbb-4ece-9c45-407a6e25a1db None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.243123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.243777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.244096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.244371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.245088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:42.245392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.246753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.258620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 251/508] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:42.259385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.281505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3c61-e87f-4beb-804d-46ca5c138d57 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.314624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.315405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.315685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.316000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.316893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.326468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.327966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:42.345839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:42.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:42.347586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:42.347586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:42.347586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:42.347586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:42.347586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 258/509] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:42.348183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.362572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332263da-3a40-46a1-bbe9-6958c4b2b755 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.396341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.396991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.397273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.398094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.398812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:42.399116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.400498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.402179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.411668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 252/510] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:42.412388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.446582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01081cdf-acc7-4370-a56d-88a2b97a6eb1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.480272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.481136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.481426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.481703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.489043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.490479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:42.504367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:42.505318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 259/511] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:42.511357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.512033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dcce0-3312-4caa-ab2a-1cbb091cc8be None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.530592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.544710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.545499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.545782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.546058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.557885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.559345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:42.576699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:42.577627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:42.578470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:42.578470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:42.578470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:42.578470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:42.578470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 253/512] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:42.578744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.608727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc3b03-17a9-4461-aab7-158071d39127 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.622465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.642923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.643716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.644033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.644316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.651694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.653178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:42.667569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:42.668459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:42.669359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 260/513] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:42.669911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.679709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88090811-ab86-4fe1-ae72-93578862dc0f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.712466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.713065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.713354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.713626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.714349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:42.714926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.716324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.727868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 254/514] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:42.728640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.759021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.773832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21645cd2-27af-4324-9d25-4c026ec50fc7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.807092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.807926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.808202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.808483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.815746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.817214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.825987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8af438-c595-4d5e-9cd1-bab55193b6c5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:42.831570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:42.832706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 261/515] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:42.834165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.862075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.862734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.863021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.863297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.864179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:42.864491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.865925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.867312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:42.904074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 255/516] 10.222.0.62 () {56 vars in 1137 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:42.904908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.939336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4040d-6c9a-4964-9ce5-2e530ae934b7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.971391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.972024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.972317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.972596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.973322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:42.973636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.974999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.986923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 262/517] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:42.987513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.999476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.009756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f556d01-8a71-4264-9ff6-99acd29044ba None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.043361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.044053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.044336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.044645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.045482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:43.045791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.047225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.064689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 256/518] 10.222.0.62 () {56 vars in 1139 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:43.065755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.079493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.086577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365ba84-fcf9-4249-b2f0-fb8b50e80925 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.120302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.121088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.121375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.121654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.128261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.129701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:43.143749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:43.144621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:43.145476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 263/519] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:43.145970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.166873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473592e4-8070-47bd-a91b-d2f6d93edbf9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.203685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.204383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.204662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.204934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.205768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:43.206075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.207539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.224733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 257/520] 10.222.0.62 () {56 vars in 1137 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:43.225962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.233930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.246366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da57378-1e5f-47d5-9203-f935195309c4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.278763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.279408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.279691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.279997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.303288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.325172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61870-6be3-496b-aed6-2b224bb9dfa7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.347294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.348877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.358920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.359580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.359903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.360185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.360887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.362299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.381096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 258/521] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:10:43 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:43.381917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.454078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.463179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 264/522] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:43.464305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.501737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0558a-3c35-411a-bd37-512e43219ed3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.528510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.541208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.542208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.542492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.542770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.582616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cfae9-034d-4353-a9eb-95e754451a55 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.621134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.622629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.623458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.624540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.625004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.625521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.626335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:43.626709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.628223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.648550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 265/523] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:43.649248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.687893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.729990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 259/524] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:43.731157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.767442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473aecea-95d0-4f21-aa61-5715c25cad15 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.784653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.805894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.806539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.806828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.807107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.842888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf52aa6-71ee-402c-b7ee-5790c501d146 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.878063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.878966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.879656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.879759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.880056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.880331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.891309 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.892814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:43.918162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:43.919314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:43.920215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:43.920215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:43.920215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:43.920215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:43.920215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 260/525] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:43.920497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.939554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.998396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 266/526] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:43.998396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.015040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.033509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73488d-73e9-4dbf-a2ad-a0e61ee3850f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.069018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.070216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.070498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.070771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.071603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=edc938bed535461e974daf01bb99e9a6) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.071946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.073389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.079395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 261/527] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/role_assignments?user.id=edc938bed535461e974daf01bb99e9a6&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.088915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.109222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5265-ace8-48b7-a387-33077aa2afa1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.154907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.156034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.156324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.156604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.172261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.174037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:44.199467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:44.200562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:44.201436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:44.201436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:44.201436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:44.201436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:44.205392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 267/528] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:44.206355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.212100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.242363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b1b5-12cd-4a71-92f7-6def39f1d099 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.292323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.293157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.293834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.294110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.295116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.295425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.297229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.297620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.314262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 262/529] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.315095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.338504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e13ebc-89e2-4a87-a6e6-e6d8aa1e4fa9 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.374397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.375302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.375582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.375884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.395850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.395850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:44.411887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:44.412766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:44.414151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:44.414151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:44.414151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:44.414151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:44.414692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 268/530] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:44.415726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.444715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9306a4d-118c-44ab-9bad-0ce52b2f8f0e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.491749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.492930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.493492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.493778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.494516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.494979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.496979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.507037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.515362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 263/531] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.516310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.548211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1f22a-a18c-49fc-8b39-8faf5e3a9353 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.567141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.585585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.586543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.586666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.586937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.594731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.596217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:44.610808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:44.611701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:44.612608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:44.617340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 269/532] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:44.618185 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.639242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37f08-da50-4777-9ae9-fec0275209a4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.677893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.679567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.680141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.680630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.681646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.682471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.684641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.698214 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 264/533] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.699296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.722499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f09c6-4841-4f5d-862e-a03e19dfbab0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.757616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.758446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.758736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.759021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.765952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.766382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.767854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:44.783563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:44.784472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:44.785760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 270/534] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:44.786232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.813388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013082d3-795e-4c70-81d3-4c47f3def89f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.826419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.852417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.853080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.853359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.853630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.854430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.854739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.856445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.875609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 265/535] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.876467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.894979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c7bc0-77b3-473b-b87b-5d3f9b3739f7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.931429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.932270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.932561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.932845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.940106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.941605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:44.956233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:44.957322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:44.958338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 271/536] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:44.958803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.002179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c50e5b-4e02-439f-8f1d-2ef9f83375b9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.021621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.042314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.043315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.043860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.044275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.045207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.045672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.047461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.058653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.061408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 266/537] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.062357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.062724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03323622-39f1-44df-973c-d9cc20e594d0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.098908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.099573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.099889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.100171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.101579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=7be3a546aa644281a0d3594395bbdf33) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.101898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.103342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.116197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 272/538] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/role_assignments?user.id=7be3a546aa644281a0d3594395bbdf33&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.117363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.177417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1c4c9-247d-48fc-81ce-51a00a969a9e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.217057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.217959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.218381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.218843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.219937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.220387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.222078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.235688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 267/539] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.241251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.290013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.298747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78a6d-9389-40c5-aec6-9a863e349bc7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.344872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d33ac-d941-44e5-a35d-c452c99236e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.345540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d33ac-d941-44e5-a35d-c452c99236e1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.345829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d33ac-d941-44e5-a35d-c452c99236e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.346110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d33ac-d941-44e5-a35d-c452c99236e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.347476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d33ac-d941-44e5-a35d-c452c99236e1 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=a364b6179abb4ad0883898090acef827) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.347845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d33ac-d941-44e5-a35d-c452c99236e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.349728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d33ac-d941-44e5-a35d-c452c99236e1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.357788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 273/540] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/role_assignments?user.id=a364b6179abb4ad0883898090acef827&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.369886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa522da0-1212-4d9e-96a0-02937b1d8a91 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.453861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c831e86-ec9b-4ca4-8a87-c83fadf646ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.454570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c831e86-ec9b-4ca4-8a87-c83fadf646ef None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.454857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c831e86-ec9b-4ca4-8a87-c83fadf646ef None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.455136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c831e86-ec9b-4ca4-8a87-c83fadf646ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.456072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c831e86-ec9b-4ca4-8a87-c83fadf646ef None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=4a8b4111f8774b03815a525fdbdaa4e9) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.456394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c831e86-ec9b-4ca4-8a87-c83fadf646ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.457938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c831e86-ec9b-4ca4-8a87-c83fadf646ef None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.469988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 268/541] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/role_assignments?user.id=4a8b4111f8774b03815a525fdbdaa4e9&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.643699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b5b14-a639-451b-b8db-8c2e0776e900 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.644068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b5b14-a639-451b-b8db-8c2e0776e900 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.644242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b5b14-a639-451b-b8db-8c2e0776e900 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.645820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 274/542] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:45.652499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74119a-c475-42a6-8d6f-2c49372b86ad None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.652815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74119a-c475-42a6-8d6f-2c49372b86ad None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.653066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74119a-c475-42a6-8d6f-2c49372b86ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.676287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9c74119a-c475-42a6-8d6f-2c49372b86ad None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:45.677508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9c74119a-c475-42a6-8d6f-2c49372b86ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:45.772853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 269/543] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:45.807722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b5b14-a639-451b-b8db-8c2e0776e900 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.861499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.974937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b5b14-a639-451b-b8db-8c2e0776e900 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:46.020572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:46.021373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.021659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.021937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.043559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:46.051290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:46.052249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:46.053041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 275/544] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:46.059181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74119a-c475-42a6-8d6f-2c49372b86ad None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:46.094475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:46.164015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74119a-c475-42a6-8d6f-2c49372b86ad None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:46.199923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:46.200799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.201078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.201351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.202108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:46.202413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:46.203795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:46.212999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 270/545] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:46.218714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:46.320740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:46.342138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e910f39-8f4d-4834-bcee-4ef0ec67c7c4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:46.396519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:46.397360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.397648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.397930 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.412588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:46.414099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:46.435114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:46.436054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:46.436916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:46.436916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:46.436916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:46.436916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:46.439929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 276/546] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:46.443035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:46.629872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:46.638733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416aa18-3213-49a2-b0d7-623fe206a5d6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:46.687588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9e82a-6655-483a-8a5b-d9bd0d7c6713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:46.688318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e82a-6655-483a-8a5b-d9bd0d7c6713 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.688607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e82a-6655-483a-8a5b-d9bd0d7c6713 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.688890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e82a-6655-483a-8a5b-d9bd0d7c6713 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.689648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9e82a-6655-483a-8a5b-d9bd0d7c6713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:46.689968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9e82a-6655-483a-8a5b-d9bd0d7c6713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:46.691419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9e82a-6655-483a-8a5b-d9bd0d7c6713 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:46.706194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 271/547] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:46.716987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:46.875753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c0a67-cc83-4cf8-bc5c-c6d06dcf5b8f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:46.915304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db8c2-7761-406c-9a81-62bba254aa40 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.915978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db8c2-7761-406c-9a81-62bba254aa40 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.916544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db8c2-7761-406c-9a81-62bba254aa40 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.918538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 272/548] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:46.924899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dba11-4f20-4d4b-8ced-40acbbf6324f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.925517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dba11-4f20-4d4b-8ced-40acbbf6324f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.926059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dba11-4f20-4d4b-8ced-40acbbf6324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.926413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:46.927235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.927521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.927800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.944878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:46.946324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:46.949434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:46.961393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-298dba11-4f20-4d4b-8ced-40acbbf6324f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:46.962867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-298dba11-4f20-4d4b-8ced-40acbbf6324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:46.971886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:46.973345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:46.974425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 277/549] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:46.977252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.075382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 273/550] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:10:47.094286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c81eb6-fa44-4a92-8732-8bf1571c7c26 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.111481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dba11-4f20-4d4b-8ced-40acbbf6324f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.139690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.140624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.141061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.142366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.143363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:47.144533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:47.148179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:47.183492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 278/551] 10.222.0.62 () {56 vars in 1135 bytes} [Fri May 27 03:10:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:47.188828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.234069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:47.253727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dba11-4f20-4d4b-8ced-40acbbf6324f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.314161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.314962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.315246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.315529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.320055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:47.332853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:47.340888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:47.342104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:47.343203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:47.344100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 274/552] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:47.352766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.406654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4805f9-7e05-46cf-8d1c-211fec35419e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.481240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.481833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.482109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.482384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.512009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1297cf2-2447-43e9-9537-2d2b124fc1fc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.573220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.573898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.574187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.574464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.575297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:47.575612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:47.577030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:47.595161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 275/553] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:47.603049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.612741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:47.620654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:47.622123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:47.739232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:47.783001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792e103-b65d-40c5-bfde-21cc05f47a14 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.809720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 279/554] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:10:47.815506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.832546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.833368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.833658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.833932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.856510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:47.858043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:47.902024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:47.903039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:47.903923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:47.903923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:47.903923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:47.903923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:47.908108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 276/555] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:47.914601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.944291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976ae5-2dd4-4056-9e1b-d6d143c73fea None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.987627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad41d0-9ee0-4822-b913-4464e49dd0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.989742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad41d0-9ee0-4822-b913-4464e49dd0ee None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.990260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad41d0-9ee0-4822-b913-4464e49dd0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.991411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad41d0-9ee0-4822-b913-4464e49dd0ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.006714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad41d0-9ee0-4822-b913-4464e49dd0ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.009601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad41d0-9ee0-4822-b913-4464e49dd0ee None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.010628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-a3ad41d0-9ee0-4822-b913-4464e49dd0ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:48.035133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.036103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.037558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:48.037558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:48.037558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:48.037558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:48.037558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 280/556] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:48.043652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd8dc2-2575-4047-a8e3-46d7af0b7f57 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.043652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd8dc2-2575-4047-a8e3-46d7af0b7f57 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.043652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd8dc2-2575-4047-a8e3-46d7af0b7f57 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.043652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 281/557] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:48.046579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd8dc2-2575-4047-a8e3-46d7af0b7f57 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.060564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.068718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12593eb-eab3-459f-9c18-a18e9e469a48 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.107887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a51645-f365-4c45-893c-c491e1e8d530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.108548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a51645-f365-4c45-893c-c491e1e8d530 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.108843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a51645-f365-4c45-893c-c491e1e8d530 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.109130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a51645-f365-4c45-893c-c491e1e8d530 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.109866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a51645-f365-4c45-893c-c491e1e8d530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.110176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a51645-f365-4c45-893c-c491e1e8d530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.111550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a51645-f365-4c45-893c-c491e1e8d530 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.123621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 277/558] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:48.124853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821cc8b-eef1-4bcb-8f5d-1e4d4df3dfce None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.125133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821cc8b-eef1-4bcb-8f5d-1e4d4df3dfce None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.125408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821cc8b-eef1-4bcb-8f5d-1e4d4df3dfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.142818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c821cc8b-eef1-4bcb-8f5d-1e4d4df3dfce None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:48.144025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c821cc8b-eef1-4bcb-8f5d-1e4d4df3dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:48.175577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd8dc2-2575-4047-a8e3-46d7af0b7f57 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.210160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4edc0-742e-4a20-ad6e-09674c5ee819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.210747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 278/559] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:48.210926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4edc0-742e-4a20-ad6e-09674c5ee819 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.211332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4edc0-742e-4a20-ad6e-09674c5ee819 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.211679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4edc0-742e-4a20-ad6e-09674c5ee819 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.212142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d35ed-a510-4465-b8b0-ff181191b786 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.212434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d35ed-a510-4465-b8b0-ff181191b786 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.212530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4edc0-742e-4a20-ad6e-09674c5ee819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.212711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d35ed-a510-4465-b8b0-ff181191b786 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.212891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4edc0-742e-4a20-ad6e-09674c5ee819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.214098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 279/560] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:48.214199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4edc0-742e-4a20-ad6e-09674c5ee819 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.216847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d35ed-a510-4465-b8b0-ff181191b786 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.230684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 282/561] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:48.232047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090345af-677a-4507-8736-677f0fce9a8b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.232335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090345af-677a-4507-8736-677f0fce9a8b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.232611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090345af-677a-4507-8736-677f0fce9a8b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.233996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 283/562] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:48.234864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624c84b-bfa1-4ec7-ba40-dffa7dc2dbd4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.235147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624c84b-bfa1-4ec7-ba40-dffa7dc2dbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.235423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624c84b-bfa1-4ec7-ba40-dffa7dc2dbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.251588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-6624c84b-bfa1-4ec7-ba40-dffa7dc2dbd4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:48.252728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6624c84b-bfa1-4ec7-ba40-dffa7dc2dbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:48.271707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.313696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.320782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 284/563] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:48.321580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624c84b-bfa1-4ec7-ba40-dffa7dc2dbd4 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.323478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d35ed-a510-4465-b8b0-ff181191b786 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.359995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c008405-d4f1-43d7-a159-0e2c2ef77d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.360791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008405-d4f1-43d7-a159-0e2c2ef77d6d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.361077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008405-d4f1-43d7-a159-0e2c2ef77d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.361354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008405-d4f1-43d7-a159-0e2c2ef77d6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.368310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c008405-d4f1-43d7-a159-0e2c2ef77d6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.369756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c008405-d4f1-43d7-a159-0e2c2ef77d6d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-5c008405-d4f1-43d7-a159-0e2c2ef77d6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:48.385733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.386711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:48.387724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 280/564] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:48.388355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73c2b-8b55-4b22-8926-3f93c47c8b7b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.388355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73c2b-8b55-4b22-8926-3f93c47c8b7b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.388775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73c2b-8b55-4b22-8926-3f93c47c8b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.404837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-71e73c2b-8b55-4b22-8926-3f93c47c8b7b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:48.406015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-71e73c2b-8b55-4b22-8926-3f93c47c8b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:48.421721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624c84b-bfa1-4ec7-ba40-dffa7dc2dbd4 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.454760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.455539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.455851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.456126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.463210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.464923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.470400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 281/565] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:48.471216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e73c2b-8b55-4b22-8926-3f93c47c8b7b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:48.478540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.479442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 285/566] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:48.480763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.516887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.537536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.579651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae6419-a608-4562-8452-83832257ca4f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.597079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e73c2b-8b55-4b22-8926-3f93c47c8b7b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.618659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.619455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.619741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.620044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.629818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:48.631149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.632567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.632850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.633125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:48.637966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.639039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 286/567] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:48.640539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.648810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:48.656534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 282/568] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:48.658873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.742575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602463a3-3f55-408a-8a86-64225ee5a0c5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.755500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.759857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.760301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afc9a3-c4b8-4e00-83ae-6506dd3c725c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.777450 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.778047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.778334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.778617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.779461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.779788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.781242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.794954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.795600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.795907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.796185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.796962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.797269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.798375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 287/569] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:48.798678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.799168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.815112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 283/570] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:48.815112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.901743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2ca31-0f5b-4128-bb58-1db5fd393d67 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.910621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ff6f7-935b-4c82-b06d-ca7e4e7901dd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.936036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.936831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.937113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.937391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.947427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.947991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:48.948409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.948409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.948662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.949414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.949726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.951413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:48.956144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.957229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 288/571] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:48.963775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.965788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 284/572] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:48.965788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.986574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.994395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.058464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5d07b-df8d-4e23-a349-3bc15757e98a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.066459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856f814-6799-4e55-964d-db4b46dddc76 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.091437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.092124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.092410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.092687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.093437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.093746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.095106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.101709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.102568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.102824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 289/573] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:49.102960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.103249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.103596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.114524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.116303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:49.134035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.135049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.135974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:49.135974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:49.135974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.135974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:49.135974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 285/574] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:49.136684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.197768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.201900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baff45-f231-404c-ad44-ceeade2e981b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.208541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.235319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.236000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.236284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.236566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.237400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=4b633648662a4b51bad083fd4637f204) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.237708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.239117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.239445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9355c7c-3d2c-4de3-a569-05c5c21292f5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.244419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 290/575] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/role_assignments?user.id=4b633648662a4b51bad083fd4637f204&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:49.245181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.287992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.288932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.289458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.289826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.303026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.305085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:49.325043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.325993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.326883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:49.326883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:49.326883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.326883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:49.326883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 286/576] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:49.327159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.357812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9807-3fca-4c8a-b4ca-2372430b43e1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.395114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.395807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.396122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.396405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.397237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.397553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.398954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.408328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 291/577] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:49.409278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.442529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.455551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcec58c-7357-4fd9-ad3e-1d4c5a953f01 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.481756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.494823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.495020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.495307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.495589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.508403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.509872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.525797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aea129-fac9-4f12-a0f5-e4c935b842fc None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:49.527570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.528476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.534125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:49.534125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:49.534125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.534125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:49.534125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 287/578] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:49.534125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.569529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.570193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.570481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.570762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.571496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.571809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.573215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.590699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 292/579] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:49.590699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.638580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a745b5-fffb-43e8-acfb-3d9d84990032 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.672963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.673623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.673906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.674183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.674854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.674973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.675213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.676871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.693563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 288/580] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:49.693563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.698803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c168f-04be-4558-bdd5-e37ea3145fc7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.731960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.743863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.745140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.745970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.746531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.757362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.758833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:49.778598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.781170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:49.781170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:49.781170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.781170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:49.781170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 293/581] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:49.781462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.807593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5e0e4-2250-4099-99cb-215a722e34bc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.842189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.842938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.843062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.843312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.844072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.844383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.845745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.861678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 289/582] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:49.862450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.890658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5d0c0-bc48-4ac2-a1fd-9cf17e28b46f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.933076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.933076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.933076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.933076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.938415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.939607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.941432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:49.958891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.960376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 294/583] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:49.961987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.970935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.978142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bbbcd-994f-4e16-bd84-b8cb0f1aa9ca None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.012962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.013766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.014054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.014333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.022628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.024107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:50.038871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:50.040179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:50.041180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 290/584] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:50.041800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.084981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99222d-6ab6-450b-ad7e-ad33f30be830 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.118081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.118667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.118950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.119223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.119992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:50.120301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.121661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.136211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 295/585] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:50.136846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.152558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae02fd-d62a-45d4-97c2-e41ecb3326ff None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.187501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.188541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.188984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.189267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.196079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.197013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.198482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:50.213098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:50.214051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:10:50.215275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:50.219120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 291/586] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:50.219943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.241400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.243380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead3ce6-032b-4f58-865e-35c7d2f56e6f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.284678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.285347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.285632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.285909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.286704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:50.287015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.289035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.300663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 296/587] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:50.301463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.343035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d4e2-e000-4998-bae8-da6e6778b069 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.381564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.382541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.382824 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.383103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.383979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:50.384288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.385717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.404633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 292/588] 10.222.0.62 () {56 vars in 1139 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:50.405245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.412166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b8fe8-2b67-4d73-ab91-be218b14fe8c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.430069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.450296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.451090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.451485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.451765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.460377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.461950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:50.477316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:50.478347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 297/589] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:50.479999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.485266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.518548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0d9d-27d4-4581-9dcb-e71e089d7508 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.554561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.555334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.555620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.555936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.556775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:50.557093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.558626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.577840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 293/590] 10.222.0.62 () {56 vars in 1137 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:50.577840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.592365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e8e8-89be-436b-a9c3-1d2504f7cae3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.629871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.629871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.629871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.629871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/edc938bed535461e974daf01bb99e9a6/roles/cf8ecb4f65d74cccbc07ab4e466a76ad` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.644464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=edc938bed535461e974daf01bb99e9a6, role_id=cf8ecb4f65d74cccbc07ab4e466a76ad)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.645922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.659767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 298/591] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:50 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/edc938bed535461e974daf01bb99e9a6/roles/cf8ecb4f65d74cccbc07ab4e466a76ad => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:50.660591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.696640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.703475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76c7e2-cc8d-4809-98d2-ec2a43becf88 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.749616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.764559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.765380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.765663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.765942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.784097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.786972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.794743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac558e-a0fb-4a0e-9091-aaba7d5be8ba None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:50.813133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:50.814112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:50.814986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:50.814986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:50.814986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:50.814986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:50.817735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 294/592] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:50.819501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.844100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.844964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.845443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.845824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.846827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:50.847257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.848879 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.871911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 299/593] 10.222.0.62 () {56 vars in 1137 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:50.872971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.958237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404c82f-4b42-424d-a33d-43bf8851ece3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.997761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.999146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.999988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.000305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.000584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.013862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.015483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.017540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a8c3e-9079-48be-81ff-0a0c9cbe80bd None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:51.034712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:51.035633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:51.037129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:51.037129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:51.037129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:51.037129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:51.037129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 295/594] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:51.037129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.060198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.060869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.061154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.061431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.062168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:51.062480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.063875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.076920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 300/595] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:51.077765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.086806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.177210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a9d8-caa9-4577-8c9a-24d344f80ffc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.184290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f71064-6232-4ee7-8195-1872772aa9f3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.217728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.218913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.219255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.219535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.224571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.225993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.226285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.226566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.227866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:51.228186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.230400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.231050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.232631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.250549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 301/596] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:51.251524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:51.251839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:51.253844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:51.255933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:51.255933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:51.255933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:51.255933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:51.259376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 296/597] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:51.259376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.261642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.359158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.373468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9c8fc-f1bd-420e-9a3e-df84e3bf3824 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.395526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501fcf-222d-46fc-bddc-621922c9372f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.412197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.414033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.414476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.415143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.424976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.427436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.434178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.435070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.435397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.435740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.442908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.444560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:51.445717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:51.446758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:51.447866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:51.450319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 297/598] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:51.451247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:51.462414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:51.463376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:51.464576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 302/599] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:51.465542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.535260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.567204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17b571-6e03-4e98-85ae-d454c8c4ad9f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.578493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d3133-c493-4ba4-8c20-b386ca434cd2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.603878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc013d-b8eb-4949-a60e-c36417d24a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.604580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc013d-b8eb-4949-a60e-c36417d24a0b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.604865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc013d-b8eb-4949-a60e-c36417d24a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.605143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc013d-b8eb-4949-a60e-c36417d24a0b None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.605868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc013d-b8eb-4949-a60e-c36417d24a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:51.606179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc013d-b8eb-4949-a60e-c36417d24a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.607550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc013d-b8eb-4949-a60e-c36417d24a0b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.619644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.620590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.621077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.621429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 298/600] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:51.621791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.622805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648ec34-b192-4c4d-a5bc-658432ac16f9 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.623107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648ec34-b192-4c4d-a5bc-658432ac16f9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.623378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648ec34-b192-4c4d-a5bc-658432ac16f9 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.624136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:51.624887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 299/601] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:51.625172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.625575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648ec34-b192-4c4d-a5bc-658432ac16f9 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.627356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.638771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.640548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 303/602] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:51.641508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.737209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648ec34-b192-4c4d-a5bc-658432ac16f9 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.753084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbaaf8-8ff3-4431-8b0a-e32fce2aa53e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.771522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debf862-bbc8-4cdb-aa01-5f0c729d778e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.772713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debf862-bbc8-4cdb-aa01-5f0c729d778e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.772997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debf862-bbc8-4cdb-aa01-5f0c729d778e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.773270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debf862-bbc8-4cdb-aa01-5f0c729d778e None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.774068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debf862-bbc8-4cdb-aa01-5f0c729d778e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:51.774373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debf862-bbc8-4cdb-aa01-5f0c729d778e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.775804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debf862-bbc8-4cdb-aa01-5f0c729d778e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.778044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.793343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 300/603] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:51.793343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bd68d-b896-401a-9334-841a43568a37 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.793343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bd68d-b896-401a-9334-841a43568a37 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.793343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bd68d-b896-401a-9334-841a43568a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.795399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.796237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.796525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.796801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.804572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:51.806159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:51.808756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-057bd68d-b896-401a-9334-841a43568a37 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:51.809700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-057bd68d-b896-401a-9334-841a43568a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:51.820105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:51.821012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:51.821924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 304/604] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:51.822448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.882238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.889097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 301/605] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:51.889902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bd68d-b896-401a-9334-841a43568a37 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.946783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8d1b-7020-499c-976e-4a15e377b21d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.984653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.985662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.985947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.986337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/7be3a546aa644281a0d3594395bbdf33/roles/cf8ecb4f65d74cccbc07ab4e466a76ad` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.994861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bd68d-b896-401a-9334-841a43568a37 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.001891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=7be3a546aa644281a0d3594395bbdf33, role_id=cf8ecb4f65d74cccbc07ab4e466a76ad)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.003466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.027851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 305/606] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:51 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/7be3a546aa644281a0d3594395bbdf33/roles/cf8ecb4f65d74cccbc07ab4e466a76ad => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:52.027851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.043459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.044443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.044796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.045140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/a364b6179abb4ad0883898090acef827/roles/cf8ecb4f65d74cccbc07ab4e466a76ad` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.065798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.068003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=a364b6179abb4ad0883898090acef827, role_id=cf8ecb4f65d74cccbc07ab4e466a76ad)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.071650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.091489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 302/607] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:51 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/a364b6179abb4ad0883898090acef827/roles/cf8ecb4f65d74cccbc07ab4e466a76ad => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:52.092588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.152564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.159360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5008a4-1433-4d2e-98d2-c3c5f3599c0d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.206373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.208130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.208602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.209800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.211327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:52.212018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.214053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.218289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf896-431d-44ef-9198-0e950b3170f0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.230910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 306/608] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:52.242800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.287068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.288041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.288330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.288606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.302398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:52.311380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:52.312558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:52.313408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 303/609] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:52.318623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.411613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.418439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31699e7e-3ee5-4b48-80c9-7541f5c7ffc3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.446565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091f398-1b21-4f62-8b6c-e53eedb8c28b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.455947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.456460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.457474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.457762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.458041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/4a8b4111f8774b03815a525fdbdaa4e9/roles/cf8ecb4f65d74cccbc07ab4e466a76ad` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.474771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=4a8b4111f8774b03815a525fdbdaa4e9, role_id=cf8ecb4f65d74cccbc07ab4e466a76ad)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.476393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.496930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.497807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.498108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.498371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.499148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:52.499445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.503863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 307/610] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:52 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/4a8b4111f8774b03815a525fdbdaa4e9/roles/cf8ecb4f65d74cccbc07ab4e466a76ad => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:52.504084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.525052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 304/611] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:52.530891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.822034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfabc07-b6d4-49fe-94e2-46a406d7a997 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.925642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0c360-e0a3-46ae-9bc0-17556d8314f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.926478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c360-e0a3-46ae-9bc0-17556d8314f0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.926764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c360-e0a3-46ae-9bc0-17556d8314f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.927043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c360-e0a3-46ae-9bc0-17556d8314f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.958314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0c360-e0a3-46ae-9bc0-17556d8314f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.960397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0c360-e0a3-46ae-9bc0-17556d8314f0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.965122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-29c0c360-e0a3-46ae-9bc0-17556d8314f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:52.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:52.983411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:52.984719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:52.984719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:52.984719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:52.984719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:52.984719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 308/612] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:52.989274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.150240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a2ec-93e5-4d34-b3ba-b2bf4eb72a93 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.174784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672710-9996-4cab-8614-9a51238f79a5 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.175306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672710-9996-4cab-8614-9a51238f79a5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.175306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672710-9996-4cab-8614-9a51238f79a5 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.176721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 309/613] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:53.183225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c18fa-2222-4651-8c2e-429a4d75c00d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.183522 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c18fa-2222-4651-8c2e-429a4d75c00d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.183847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c18fa-2222-4651-8c2e-429a4d75c00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.208481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c59c18fa-2222-4651-8c2e-429a4d75c00d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:53.209746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c59c18fa-2222-4651-8c2e-429a4d75c00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:53.235845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.235845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.235845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.235845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.236584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:53.236584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.236896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.264000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 305/614] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:53.273761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.314992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 310/615] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:53.328646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:53.376448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c18fa-2222-4651-8c2e-429a4d75c00d None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.454043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea80dc-c9a2-45bd-9aac-670e9227df1c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.522788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.523597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.523912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.524195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.531972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.533421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:53.550020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:53.551018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:53.552014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 306/616] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:53.558515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.565025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c18fa-2222-4651-8c2e-429a4d75c00d None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.598080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:53.609729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.610882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.611363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.612960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.635376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:53.655742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:53.656764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:53.657622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:53.660325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 311/617] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:53.668315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.732287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:53.737232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a4ee-af6d-4a41-850d-f4a8f73ee6fc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.774438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.775465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.775995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.776481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.777514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:53.778109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.779734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 307/618] 10.222.0.62 () {56 vars in 1135 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:53.807011 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.862779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341500-2061-4da3-a41c-253f3775ef9b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.907589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1007a-9f4d-40af-9cfe-f48f1a88b5ef None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.913708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.914373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.914665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.914943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.922342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:53.922676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.924393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.943150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 312/619] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:53.944070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.996602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.997325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.997614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.997892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.000062 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.018012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.019496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.042029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:54.051511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:54.052460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:54.054040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:54.054040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:54.054040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:54.054040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:54.058112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 308/620] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:54.064660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.088925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63539a-138a-42cb-b775-b0dad7684eac None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.153441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.154241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.154528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.154809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.177287 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.181137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:54.211574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:54.213727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:54.216101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:54.216101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:54.216101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:54.216101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:54.216101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 313/621] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:54.223406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.237842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54849554-5d62-445e-96c9-f3eba5ab22c2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.278413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.279288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.279578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.279878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.280883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:54.281617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.283614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.296755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 309/622] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:54.303766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.362604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.365645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3dd40-e774-4561-9299-9969f1b53e8a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.401879 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.402566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.402858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.403146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.403927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:54.404249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.405723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.419107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 314/623] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:54.424653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.442252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.527980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac4baa-5c21-4c47-b07f-b4580d371073 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.531677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6726922-f1aa-497e-9546-862f4e907957 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.572311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.573510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.573945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.574374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.583692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.585316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.585752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.586141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.586425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.586710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.594951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:54.603383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:54.604503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:54.605493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:54.607898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.608322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 315/624] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:54.613947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:54.631492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:54.632500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:54.633413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:54.641115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 310/625] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:54.644181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.648734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.776597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28b548-b5ae-4746-bd73-2e306b0af758 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.813885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7232544-d278-42b4-9038-af3b5bf20724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.814482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7232544-d278-42b4-9038-af3b5bf20724 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.814771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7232544-d278-42b4-9038-af3b5bf20724 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.815049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7232544-d278-42b4-9038-af3b5bf20724 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.815933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7232544-d278-42b4-9038-af3b5bf20724 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:54.816248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7232544-d278-42b4-9038-af3b5bf20724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.817673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7232544-d278-42b4-9038-af3b5bf20724 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.829480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95760c1-1b34-40de-ae65-f36186c623f3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.837166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 316/626] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:54.838487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ca1be-fcec-44c6-be2a-90f20ee0acab None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.838775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ca1be-fcec-44c6-be2a-90f20ee0acab None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.839306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ca1be-fcec-44c6-be2a-90f20ee0acab None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.845314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 317/627] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:54.849798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ca1be-fcec-44c6-be2a-90f20ee0acab None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.874743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c3bd8-1761-456d-978c-0aea1fe43351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.875415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3bd8-1761-456d-978c-0aea1fe43351 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.875709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3bd8-1761-456d-978c-0aea1fe43351 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.876024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3bd8-1761-456d-978c-0aea1fe43351 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.877107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3bd8-1761-456d-978c-0aea1fe43351 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:54.877405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3bd8-1761-456d-978c-0aea1fe43351 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.880162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3bd8-1761-456d-978c-0aea1fe43351 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.883121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.895381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 311/628] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:54.896711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738a80-f6c3-4bdb-85a7-3f7d1793641e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.897038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738a80-f6c3-4bdb-85a7-3f7d1793641e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.897316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738a80-f6c3-4bdb-85a7-3f7d1793641e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.914520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2a738a80-f6c3-4bdb-85a7-3f7d1793641e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:54.918365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2a738a80-f6c3-4bdb-85a7-3f7d1793641e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:54.945043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.969107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ca1be-fcec-44c6-be2a-90f20ee0acab None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.997342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 312/629] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:54.997342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f49c4-a55c-4ec8-8fa5-f8e56c89f1b9 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.997342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f49c4-a55c-4ec8-8fa5-f8e56c89f1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.997342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f49c4-a55c-4ec8-8fa5-f8e56c89f1b9 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.997342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 313/630] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:54.997342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f49c4-a55c-4ec8-8fa5-f8e56c89f1b9 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.010814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.011598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.011977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.012333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.081598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.083108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.109795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f49c4-a55c-4ec8-8fa5-f8e56c89f1b9 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.123306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.149801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2eca9-29af-471e-a8df-1e3e0eec7bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.150683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2eca9-29af-471e-a8df-1e3e0eec7bf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.150965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2eca9-29af-471e-a8df-1e3e0eec7bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.151242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2eca9-29af-471e-a8df-1e3e0eec7bf4 None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/4b633648662a4b51bad083fd4637f204/roles/cf8ecb4f65d74cccbc07ab4e466a76ad` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.164219 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.166347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2eca9-29af-471e-a8df-1e3e0eec7bf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=4b633648662a4b51bad083fd4637f204, role_id=cf8ecb4f65d74cccbc07ab4e466a76ad)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.168007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2eca9-29af-471e-a8df-1e3e0eec7bf4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.183722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 314/631] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:10:54 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/4b633648662a4b51bad083fd4637f204/roles/cf8ecb4f65d74cccbc07ab4e466a76ad => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:55.184848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a405f8-faf9-40c6-9dbf-0a6fd73e0e31 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.185136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a405f8-faf9-40c6-9dbf-0a6fd73e0e31 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.185412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a405f8-faf9-40c6-9dbf-0a6fd73e0e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.204169 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-a0a405f8-faf9-40c6-9dbf-0a6fd73e0e31 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:55.204561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 318/632] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:55.205357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a0a405f8-faf9-40c6-9dbf-0a6fd73e0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:55.206448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.295850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 315/633] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:55.295850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a405f8-faf9-40c6-9dbf-0a6fd73e0e31 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.363158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f565ae-37e4-4c56-aead-3b04c4454cdb None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.407469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.408420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.408706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.408980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.420418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:55.430787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:55.431748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:55.432874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 319/634] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:55.433773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.435007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.448450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a405f8-faf9-40c6-9dbf-0a6fd73e0e31 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.482247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.497914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.498717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.499003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.499280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.509506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.510965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:55.529447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:55.530390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:55.531291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:55.531291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:55.531291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:55.531291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:55.531291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 316/635] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:55.531572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.554748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ef94f-af4e-42b5-953a-7c0a6b38d6a4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.591493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e0baa-0a08-477e-bf52-111531cd658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.594275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e0baa-0a08-477e-bf52-111531cd658d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.594706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e0baa-0a08-477e-bf52-111531cd658d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.594991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e0baa-0a08-477e-bf52-111531cd658d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.606362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e0baa-0a08-477e-bf52-111531cd658d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-6c4e0baa-0a08-477e-bf52-111531cd658d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:55.614159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:55.615132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:55.616044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 320/636] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:55.617236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d37ec-990c-4c71-821b-84b8407dcbd9 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.617236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d37ec-990c-4c71-821b-84b8407dcbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.617470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d37ec-990c-4c71-821b-84b8407dcbd9 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.618835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 321/637] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:55.620605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d37ec-990c-4c71-821b-84b8407dcbd9 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.648829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373576bb-a151-44d4-822a-e363ae07cb73 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.686165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.686838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.687127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.687406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.688193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:55.688506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.689893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.703942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 317/638] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:55.703942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.706812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.724587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.729463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d37ec-990c-4c71-821b-84b8407dcbd9 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.764440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08f245-2b86-4ddb-96c8-70523d1c5af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.765106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08f245-2b86-4ddb-96c8-70523d1c5af6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.765395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08f245-2b86-4ddb-96c8-70523d1c5af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.765675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08f245-2b86-4ddb-96c8-70523d1c5af6 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.766498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08f245-2b86-4ddb-96c8-70523d1c5af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:55.766724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08f245-2b86-4ddb-96c8-70523d1c5af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.768152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08f245-2b86-4ddb-96c8-70523d1c5af6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.785271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 322/639] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:55.785271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeec256-d6cc-4c63-8c65-00582bafc085 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.785271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeec256-d6cc-4c63-8c65-00582bafc085 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.785271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeec256-d6cc-4c63-8c65-00582bafc085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.800675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-8aeec256-d6cc-4c63-8c65-00582bafc085 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:55.801852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8aeec256-d6cc-4c63-8c65-00582bafc085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:55.822939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de0908-b0f1-4e10-9575-d234a58b701c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.858540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.859360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.859718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.860114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.860978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:55.861355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.862738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.875731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 318/640] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:55.875731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.885321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 323/641] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:55.886872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeec256-d6cc-4c63-8c65-00582bafc085 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.947176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.986272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e77f92-bc77-428a-b146-0bcca8ad6b8f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.987469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.002806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeec256-d6cc-4c63-8c65-00582bafc085 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.022800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.023604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.023916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.024194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.035705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.037308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.040779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.041653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.042016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.042369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.049279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.050834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:56.056131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:56.056992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:56.057917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:56.057917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:56.057917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:56.057917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:56.062272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 319/642] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:56.062851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:56.065099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:56.066069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:56.066982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 324/643] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:56.069097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.170907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36633d-dc4e-41ed-bfb3-e7ce45336818 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.179018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b46265-5c41-486c-936f-dd1a80057cd7 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.188874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.208746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.209559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.209845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.210129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.214270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.215053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.215338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.215617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.221389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.222840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.226048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:56.233815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:56.234743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 325/644] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:56.235731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:56.245855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:56.246927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:56.248102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:56.248102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:56.248102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:56.248102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:56.250086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 320/645] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:56.251446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.255475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.344721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aae4b-d747-4b9a-8dd3-c0e20cc6149d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.367886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee85e2-bf55-44ba-9079-fa54636099fe None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.378105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.378760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.379040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.379318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.380071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:56.380375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.382258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.394302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 326/646] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:56.395101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.410251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.411367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.411650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.412484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.413289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:56.413601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.415058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.429910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 321/647] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:56.430664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.464992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.471130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.507265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d4dad-e1c9-453d-8256-1a3e69d4f3cb None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.543078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6adba7-06cd-4c9e-b971-2fbb488b7169 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.543509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.544209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.544491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.544766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.545512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:56.545817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.547180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.559944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 327/648] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:56.559944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.585831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.586501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.586786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.587062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.587798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:56.588125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.589838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.603119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 322/649] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:56.607802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.667587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2e916-0f75-4cd3-aa21-1d705d391736 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.697669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.703662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.704606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.705015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.705404 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.710108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.714929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.716494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.721982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c039c5-ad30-46c2-a19c-2715c5627ea1 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:56.730820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:56.732452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 328/650] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:56.733406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.759311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.759999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.760279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.760547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.761377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=edc938bed535461e974daf01bb99e9a6) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:56.761687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.763108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.779850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 323/651] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/role_assignments?user.id=edc938bed535461e974daf01bb99e9a6&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:56.779850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.888375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143626c2-a8ad-4fc0-9a73-3fd214cfd470 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.893928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9d92a-5b60-4af7-b1c3-14a6aeb2291f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.934116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.936744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.937039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.937971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.947151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.948007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.948126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.948287 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.948573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.949813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.962669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.964433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:56.968152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:56.969283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:10:56.970437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:56.972938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 324/652] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:56.973983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:56.984878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:56.985848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:56.986913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:56.986913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:56.986913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:56.986913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:56.988741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 329/653] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:56.989831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.998089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.050385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.108811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809c9ab-9976-4547-b4f3-96f4af63c9bf None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.121604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b33a2-e4b2-4efc-ab91-9c6c54ef911c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.151622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.152340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.152629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.152907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.153925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:57.154241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.156321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.167998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.168705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.168990 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.169262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.170120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:57.170435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.171886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.178518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 325/654] 10.222.0.62 () {56 vars in 1137 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:57.179645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:57.191048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 330/655] 10.222.0.62 () {56 vars in 1139 bytes} [Fri May 27 03:10:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:57.192084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:57.259946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.294759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423232a-6b48-4050-ab0b-0118f95243f0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.295448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.298345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeaffa-b07f-46c5-b87e-31b21f2d0f52 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.335924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.336739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.337127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.337504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.337670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.338604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.339294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.339708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.340590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:57.341146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.342768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.358129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 326/656] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:57.359204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:57.414652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.416344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.467998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76d3a-e1c4-42c5-b1ea-c8e7ee05f996 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.503704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.509731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.510474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.511172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.511781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.514949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.519306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 331/657] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:57.520474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:57.590489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.591987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.626177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bdc7c-40c8-4e5f-bd12-b79e016f228b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.661958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf5e81-58b4-4827-8278-8133185aede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.663058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf5e81-58b4-4827-8278-8133185aede6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.663542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf5e81-58b4-4827-8278-8133185aede6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.663833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf5e81-58b4-4827-8278-8133185aede6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.671849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf5e81-58b4-4827-8278-8133185aede6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.673286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf5e81-58b4-4827-8278-8133185aede6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-1abf5e81-58b4-4827-8278-8133185aede6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:57.688426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:57.689408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:57.690392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:57.694793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 332/658] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:57.696125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853abdb1-00a9-4b94-b5f1-746fea7e695b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.696416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853abdb1-00a9-4b94-b5f1-746fea7e695b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.696697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853abdb1-00a9-4b94-b5f1-746fea7e695b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.697998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 333/659] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:57.698689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853abdb1-00a9-4b94-b5f1-746fea7e695b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:57.707353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 327/660] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:57.708818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:57.743784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.770831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:57.815131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853abdb1-00a9-4b94-b5f1-746fea7e695b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.817233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae20e3-55a4-40b8-a88b-ee1b912c1e82 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:57.851546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.852084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca4957-719b-411a-8bc7-b22df081353c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:57.852417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.852701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.852802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4957-719b-411a-8bc7-b22df081353c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.853020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4957-719b-411a-8bc7-b22df081353c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.853132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.853294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4957-719b-411a-8bc7-b22df081353c None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.854134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca4957-719b-411a-8bc7-b22df081353c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:57.854448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca4957-719b-411a-8bc7-b22df081353c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.855897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca4957-719b-411a-8bc7-b22df081353c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.863477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:57.865637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:57.877297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 328/661] 10.222.0.62 () {56 vars in 1137 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:57.877297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576753d5-9e6c-4feb-8ab3-ee18218fe058 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:57.877297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576753d5-9e6c-4feb-8ab3-ee18218fe058 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:57.877297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576753d5-9e6c-4feb-8ab3-ee18218fe058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:57.888341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:57.889223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:57.890139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:57.890139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:57.890139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:57.890139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:57.892861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-576753d5-9e6c-4feb-8ab3-ee18218fe058 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:57.894056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-576753d5-9e6c-4feb-8ab3-ee18218fe058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:57.894375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 334/662] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:57.895195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:57.978063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 329/663] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:57.978940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576753d5-9e6c-4feb-8ab3-ee18218fe058 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.009948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.014823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624ecf3-288e-4c70-9f86-a72f0570530a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.028708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.054223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.055025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.055313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.055599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.066785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.068309 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:58.086391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:58.087326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:58.088256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:58.088256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:58.088256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:58.088256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:58.088256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 335/664] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:58.088554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.100035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576753d5-9e6c-4feb-8ab3-ee18218fe058 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.137170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.137884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.138171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.138476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.199830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92460cb-0366-4ee6-aab7-d80d4fb3433b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.213464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.215363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.235655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.236344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.236640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.236917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.237423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.237647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:58.237958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.239327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.252016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 336/665] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:58.252882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.279246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.328749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 330/666] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:10:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:58.329796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.358998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6fdd2-44fa-43d6-8759-bf45dd7929ee None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.397919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.398714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.399003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.399284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.409687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:58.417610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:58.418549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:58.419383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 337/667] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:58.420125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.440346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eecedb-5783-4150-a1a8-4895f35ec758 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.462686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.477637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.478365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.478618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.478838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.479561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:58.479899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.481243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.493028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 331/668] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:58.493714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.516530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.537043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658058d-18c0-47dd-a593-0aefe57e0f7e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.576017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.576804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.577086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.577359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.584056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.585487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.596936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c40232-555e-4c93-a957-f62ef5ddf0b3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:58.599809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:58.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:58.601672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 338/669] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:58.602600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.635771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.637036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.637464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.637891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.649640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.651359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:58.671052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:58.672181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:58.673270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:58.673270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:58.673270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:58.673270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:58.679663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 332/670] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:58.680828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.706376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a0199-e988-491f-9dc7-14dcad257191 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.714791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.745171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.745856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.746148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.746434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.747196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:58.747510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.748921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.757523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 339/671] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:58.762475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.772307 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.790030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f5267-0aca-49b4-8e24-f9765c5650a5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.825459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.826106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.826304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.826561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.833901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.835341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:58.850778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:58.851710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:58.852666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 333/672] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:58.853765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.873843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232af1f4-1c21-4556-a92d-8792fb622c92 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.907959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.908618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.908898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.909176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.909960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:58.910269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.911683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.929379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 340/673] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:58.929379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.962924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.974056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab953e0-5165-450f-a96b-0aab584a9047 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.003443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:59.012181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.012849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.013128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.013403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.014560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:59.014871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.016617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.033661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 334/674] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:59.033661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.045029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526d35b-3a03-4869-a947-97c74e2dd266 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.081387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.082191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.082478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.082754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.094272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.095768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:59.115144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:59.116085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:59.117024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:59.117024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:59.117024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:59.117024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:59.121229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 341/675] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:59.121912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.137483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa65d91-1bfe-4553-bfe5-c54275fef50f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.172585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.173240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.173537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.173792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.174575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:59.174880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.176342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.188080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 335/676] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:59.188856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.202726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:59.220221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb432db6-1adc-4568-a688-e2b0b3551ec4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.237294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:59.254023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.254757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.255114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.255464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.256412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=7be3a546aa644281a0d3594395bbdf33) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:59.256732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.258157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.263970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 342/677] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/role_assignments?user.id=7be3a546aa644281a0d3594395bbdf33&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:59.264746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.287381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95343167-d510-468c-b9f5-7171255b0580 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.324260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d871c7c-3348-4fee-bc6b-8bc3b2f47328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.325000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d871c7c-3348-4fee-bc6b-8bc3b2f47328 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.325281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d871c7c-3348-4fee-bc6b-8bc3b2f47328 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.325554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d871c7c-3348-4fee-bc6b-8bc3b2f47328 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.333329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d871c7c-3348-4fee-bc6b-8bc3b2f47328 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.335346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d871c7c-3348-4fee-bc6b-8bc3b2f47328 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-9d871c7c-3348-4fee-bc6b-8bc3b2f47328 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:59.350692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:59.351673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:10:59.353304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 336/678] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:59.353805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c525-0440-4a90-9b88-b6b2f937b83a None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.354038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c525-0440-4a90-9b88-b6b2f937b83a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.354271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c525-0440-4a90-9b88-b6b2f937b83a None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.355761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 337/679] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:59.356317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9c525-0440-4a90-9b88-b6b2f937b83a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.387408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea59384-d49a-4838-8a02-76f8ecced955 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.428420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.429251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.429506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.429784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.430526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:59.430834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.432371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.432711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:59.447176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 343/680] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:59.448051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.476294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9c525-0440-4a90-9b88-b6b2f937b83a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.494461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:59.514626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d983b-ed58-46f5-96b4-bbce157d4afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.515282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d983b-ed58-46f5-96b4-bbce157d4afe None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.515563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d983b-ed58-46f5-96b4-bbce157d4afe None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.515888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d983b-ed58-46f5-96b4-bbce157d4afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.516720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d983b-ed58-46f5-96b4-bbce157d4afe None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=a364b6179abb4ad0883898090acef827) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:59.517035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d983b-ed58-46f5-96b4-bbce157d4afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.518466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d983b-ed58-46f5-96b4-bbce157d4afe None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.527085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 338/681] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/role_assignments?user.id=a364b6179abb4ad0883898090acef827&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:59.528250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f484-d80b-46e3-94d8-aa615968a823 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.528581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f484-d80b-46e3-94d8-aa615968a823 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.528809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f484-d80b-46e3-94d8-aa615968a823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.546529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-10f5f484-d80b-46e3-94d8-aa615968a823 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:59.547726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-10f5f484-d80b-46e3-94d8-aa615968a823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:59.565882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ab093-9351-4c6e-8c5c-9692081a12df None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.618918 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.619608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.620199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.620488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.621572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:59.621893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.624402 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.641776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 344/682] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:59.642646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.643952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 339/683] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:59.648191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f484-d80b-46e3-94d8-aa615968a823 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.745444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:59.789544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd88b9b-e15d-4555-bd07-59e60b04fdd5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.791224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:59.792389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f484-d80b-46e3-94d8-aa615968a823 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:59.833656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.834289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:59.834437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.834733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.834991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.835094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:59.835316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:59.835777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:59.836672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=4a8b4111f8774b03815a525fdbdaa4e9) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:59.836986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.838431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.844443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:59.845286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 340/684] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/role_assignments?user.id=4a8b4111f8774b03815a525fdbdaa4e9&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:59.845946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:59.846174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:59.866056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:59.867263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:10:59.868612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 345/685] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:59.875470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:00.010143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec66d65-c002-4b91-9d98-bf56b1723d9e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:00.069885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:00.070586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:00.070868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:00.071142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:00.072030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:00.072346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:00.073824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:00.113057 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 346/686] 10.222.0.62 () {56 vars in 1135 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:00.117078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:00.153346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:00.206150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f47412-cfca-4511-bf35-b5a8b0f4946c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:00.251017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:00.260723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:00.261957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:00.262574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:00.262849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:00.269988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af509cc-57b0-432a-aa6e-a46715a60588 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:00.278156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:00.288542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:00.289463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:00.290302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:00.291084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 341/687] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:00.297171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:00.315368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:00.316051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:00.316345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:00.316628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:00.403842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920772f7-f9e5-4e2b-a13e-8ba110df0cae None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:00.438981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd88893-7719-4144-b831-806179039111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:00.439648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd88893-7719-4144-b831-806179039111 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:00.439975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd88893-7719-4144-b831-806179039111 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:00.440249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd88893-7719-4144-b831-806179039111 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:00.443849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd88893-7719-4144-b831-806179039111 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:00.443849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd88893-7719-4144-b831-806179039111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:00.443849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd88893-7719-4144-b831-806179039111 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:00.452632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 342/688] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:00.464492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd88893-7719-4144-b831-806179039111 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:00.470944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:00.472472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:00.503864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:00.525757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:00.575179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd88893-7719-4144-b831-806179039111 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:00.616976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:00.618538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:00.618818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:00.619096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:00.640290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:00.641747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:00.679064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:00.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:00.681936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:00.681936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:00.681936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:00.681936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:00.681936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 343/689] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:00.685513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:00.688628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 347/690] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:11:00.691118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:00.877771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb7d18-1f20-4771-ba56-5058b821c524 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:00.881778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:00.885906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dda0a7-1231-4514-b086-e94c17e63663 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:00.940668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:00.969782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:00.970459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:00.970740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:00.971015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:00.971753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:00.972082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:00.973813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:00.974693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:00.974973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:00.975244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:00.975965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:00.988231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:00.990214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.007747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 344/691] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:01.009978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:01.010941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:01.012440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:01.012440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:01.012440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:01.012440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:01.012440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 348/692] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:01.013965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.019039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.167912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae89ae2-bfbc-4f3f-905a-dc02a042e58c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.203761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.204878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.205815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.206249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.215745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.218442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:01.234415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:01.235538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:01.236544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:01.239334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 345/693] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:01.248711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.263613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecba75-010c-4040-9416-3f60f380696d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.316534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.317328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.318508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.318508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.319297 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:01.319612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.321048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.332188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.344016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 349/694] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:01.348912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.398193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ec1b9-d0e1-4822-a953-8859996cfa75 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.412635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.460982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.461925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.462374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.462799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.464091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:01.464661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.466824 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.504681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 346/695] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:01.507768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.536856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374587cd-ae16-4b6a-9454-de86eae87438 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.604832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.605911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.606361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.606786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.618935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.620738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:01.637623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:01.638548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:01.639452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:01.641934 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 350/696] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:01.652809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.656455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.673423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfc7b7-75d2-4a46-89bb-1bcbc8b82a34 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.710053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a613f-7453-462e-a6d9-7dc2b3ab7aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.710714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a613f-7453-462e-a6d9-7dc2b3ab7aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.711052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a613f-7453-462e-a6d9-7dc2b3ab7aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.711277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a613f-7453-462e-a6d9-7dc2b3ab7aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.795787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a613f-7453-462e-a6d9-7dc2b3ab7aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.797615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a613f-7453-462e-a6d9-7dc2b3ab7aa3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.803403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.813968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500577c-8477-461f-803b-1fe42d64d087 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.850551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587169c-9a75-44ed-a4da-568a449ac894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.851631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587169c-9a75-44ed-a4da-568a449ac894 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.852141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587169c-9a75-44ed-a4da-568a449ac894 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.852606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587169c-9a75-44ed-a4da-568a449ac894 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.853586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587169c-9a75-44ed-a4da-568a449ac894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:01.854158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587169c-9a75-44ed-a4da-568a449ac894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.855999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587169c-9a75-44ed-a4da-568a449ac894 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.869869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 351/697] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:01.871518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f29a7-52cb-46b2-b69d-6cd9d3f72c91 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.872112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f29a7-52cb-46b2-b69d-6cd9d3f72c91 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.872641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f29a7-52cb-46b2-b69d-6cd9d3f72c91 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.878339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 352/698] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:01.878339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f29a7-52cb-46b2-b69d-6cd9d3f72c91 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.899366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.920629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 347/699] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:11:01.922074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfcc4-5676-4e2a-9939-376e79dda053 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.922621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfcc4-5676-4e2a-9939-376e79dda053 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.923006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfcc4-5676-4e2a-9939-376e79dda053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.943295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f4edfcc4-5676-4e2a-9939-376e79dda053 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:01.944746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f4edfcc4-5676-4e2a-9939-376e79dda053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:02.011749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f29a7-52cb-46b2-b69d-6cd9d3f72c91 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.027957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 348/700] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:02.029459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfcc4-5676-4e2a-9939-376e79dda053 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.051085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.051758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.052103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.052402 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.053254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=4b633648662a4b51bad083fd4637f204) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:02.053572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.055111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.066995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 353/701] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/role_assignments?user.id=4b633648662a4b51bad083fd4637f204&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:02.066995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.105943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:02.215379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83986e98-124d-49f3-8d57-0b79eea3b46d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.253074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ea7bf-c6b8-4c50-9c2f-07535bb7a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.253968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ea7bf-c6b8-4c50-9c2f-07535bb7a75f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.254291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ea7bf-c6b8-4c50-9c2f-07535bb7a75f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.254652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ea7bf-c6b8-4c50-9c2f-07535bb7a75f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.260747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfcc4-5676-4e2a-9939-376e79dda053 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.272870 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ea7bf-c6b8-4c50-9c2f-07535bb7a75f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.274432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ea7bf-c6b8-4c50-9c2f-07535bb7a75f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.280626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-a24ea7bf-c6b8-4c50-9c2f-07535bb7a75f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:02.295290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:02.296327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:02.297308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:02.297308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:02.297308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:02.297308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:02.297308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 354/702] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:02.298087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa988f9-7246-4bae-9556-2c61899cb371 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.298377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa988f9-7246-4bae-9556-2c61899cb371 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.298659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa988f9-7246-4bae-9556-2c61899cb371 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.304684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 355/703] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:02.305470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa988f9-7246-4bae-9556-2c61899cb371 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.310210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbcbe6-1b56-4fbf-999b-7d70ce350bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.311250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcbe6-1b56-4fbf-999b-7d70ce350bcc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.311751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcbe6-1b56-4fbf-999b-7d70ce350bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.312274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcbe6-1b56-4fbf-999b-7d70ce350bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.325394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbcbe6-1b56-4fbf-999b-7d70ce350bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.328129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbcbe6-1b56-4fbf-999b-7d70ce350bcc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-aafbcbe6-1b56-4fbf-999b-7d70ce350bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:02.361735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:02.362902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:02.363839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:02.363839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:02.363839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:02.363839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:02.364407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 349/704] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:02.366125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f189-ee4d-4605-8281-b021c1b34330 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.366421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f189-ee4d-4605-8281-b021c1b34330 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.366699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f189-ee4d-4605-8281-b021c1b34330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.391117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4f92f189-ee4d-4605-8281-b021c1b34330 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:02.392353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4f92f189-ee4d-4605-8281-b021c1b34330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:02.400454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:02.448568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa988f9-7246-4bae-9556-2c61899cb371 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.473032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 350/705] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:02.476123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f189-ee4d-4605-8281-b021c1b34330 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.496523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.497194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.497523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.497801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.498815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:02.499124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.500582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.514141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 356/706] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:02.514982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.602022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:02.609333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f189-ee4d-4605-8281-b021c1b34330 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.625905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdadbb-0ed1-4557-873b-4e112d9e6dea None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.645056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.645743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.646678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.647184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.648281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:02.648708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.650547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.653565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:02.662363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbbecf-4dd4-41ce-8b09-9f915a52124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.663167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbecf-4dd4-41ce-8b09-9f915a52124e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.667719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbecf-4dd4-41ce-8b09-9f915a52124e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.668012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 351/707] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:02.668449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbecf-4dd4-41ce-8b09-9f915a52124e None admin] PATH_INFO: `/v3/services/network` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.668449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbbecf-4dd4-41ce-8b09-9f915a52124e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.668628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.668724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbbecf-4dd4-41ce-8b09-9f915a52124e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-d4cbbecf-4dd4-41ce-8b09-9f915a52124e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:02.692594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:02.694176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:02.695032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 357/708] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:02.695876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426c4c6-b70c-4bbb-85f8-56bfc13c025e None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.695876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426c4c6-b70c-4bbb-85f8-56bfc13c025e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.696173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426c4c6-b70c-4bbb-85f8-56bfc13c025e None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.702068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 358/709] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:02.702844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426c4c6-b70c-4bbb-85f8-56bfc13c025e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.780256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583efa0-c702-4070-bc0d-dfb53da31614 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.810001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426c4c6-b70c-4bbb-85f8-56bfc13c025e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.816869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.817549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.818386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.818667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.819438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.820912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.845472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e3197-e0f5-4e5a-809b-8f680d453e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.846290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e3197-e0f5-4e5a-809b-8f680d453e0e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.846578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e3197-e0f5-4e5a-809b-8f680d453e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.846856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e3197-e0f5-4e5a-809b-8f680d453e0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.848220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 352/710] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:11:02 2022] POST /identity/v3/users => generated 254 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:02.850407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.855856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e3197-e0f5-4e5a-809b-8f680d453e0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.856688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e3197-e0f5-4e5a-809b-8f680d453e0e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.868145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-8b1e3197-e0f5-4e5a-809b-8f680d453e0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:02.874344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:02.875708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:02.877050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:02.881064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 359/711] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:02.882819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa5208-2f9d-4239-a720-552c57e30b4d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.883189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa5208-2f9d-4239-a720-552c57e30b4d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.883463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa5208-2f9d-4239-a720-552c57e30b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.906383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-deaa5208-2f9d-4239-a720-552c57e30b4d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:02.907546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-deaa5208-2f9d-4239-a720-552c57e30b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:02.966662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:03.003868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5f911-f00c-4688-8aac-639f0c37f792 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:03.004068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 360/712] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:03.007228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa5208-2f9d-4239-a720-552c57e30b4d None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:03.058646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:03.059489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:03.059780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:03.060089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:03.061656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] RBAC: Adding query filter params (name=network) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:03.062021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:03.063418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:03.069485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 353/713] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:03.070520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:03.145512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa5208-2f9d-4239-a720-552c57e30b4d None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:03.166368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:03.193064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642cc9c-15b4-4693-a276-e132b6e6d198 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:03.193510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:03.194176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:03.194727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:03.195010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:03.196557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:03.196881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:03.198320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:03.217748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 361/714] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:03.218579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:03.243474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:03.244334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:03.244620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:03.244899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:03.245568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:03.247503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:03.266423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:03.267524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:03.268459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:03.269270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 354/715] 10.222.0.62 () {56 vars in 1129 bytes} [Fri May 27 03:11:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:03.269625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:03.270323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:03.333385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c7821-3783-4a4e-bed5-6b262f9b0903 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:03.372313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:03.373080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:03.373435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:03.373787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:03.374582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] RBAC: Adding query filter params (type=network) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:03.374981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:03.376477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:03.390124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 362/716] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:03.391260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:03.408037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7c9c-f9f6-4ff2-b81b-25ade24d967e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:03.449646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:03.450637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:03.451093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:03.451566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:03.452701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=edc938bed535461e974daf01bb99e9a6) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:03.453203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:03.454865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:03.462155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 355/717] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:03 2022] GET /identity/v3/role_assignments?user.id=edc938bed535461e974daf01bb99e9a6&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:03.697421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43aa5b3-3c57-47bc-9665-6e4100c2639d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:03.705383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:03.744615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c48894-6fb3-4983-a531-91d35677f47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:03.745326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c48894-6fb3-4983-a531-91d35677f47c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:03.745616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c48894-6fb3-4983-a531-91d35677f47c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:03.745898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c48894-6fb3-4983-a531-91d35677f47c None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:03.746643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c48894-6fb3-4983-a531-91d35677f47c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:03.746958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c48894-6fb3-4983-a531-91d35677f47c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:03.748489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c48894-6fb3-4983-a531-91d35677f47c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:03.754002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 363/718] 10.222.0.62 () {56 vars in 1124 bytes} [Fri May 27 03:11:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:03.759198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:03.762258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:03.870736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6d10d-d698-4c0c-87f4-df5ff590b037 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:03.905779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952d30f-862f-4b5b-a474-c2b669a8a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:03.906452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952d30f-862f-4b5b-a474-c2b669a8a016 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:03.906737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952d30f-862f-4b5b-a474-c2b669a8a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:03.907018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952d30f-862f-4b5b-a474-c2b669a8a016 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:03.907742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952d30f-862f-4b5b-a474-c2b669a8a016 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:03.908091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952d30f-862f-4b5b-a474-c2b669a8a016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:03.909457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952d30f-862f-4b5b-a474-c2b669a8a016 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:03.921450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 356/719] 10.222.0.62 () {56 vars in 1124 bytes} [Fri May 27 03:11:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:04.514935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5175c-c826-4237-94d6-49adae5cca3f None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:04.515329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5175c-c826-4237-94d6-49adae5cca3f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:04.515547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5175c-c826-4237-94d6-49adae5cca3f None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:04.517164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 364/720] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:04.522200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c573-48c8-4924-a88c-9f44c4b7dabe None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:04.522499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c573-48c8-4924-a88c-9f44c4b7dabe None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:04.522784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c573-48c8-4924-a88c-9f44c4b7dabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:04.544842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2f45c573-48c8-4924-a88c-9f44c4b7dabe None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:04.546039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2f45c573-48c8-4924-a88c-9f44c4b7dabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:04.625254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:04.638560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 357/721] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:04.673574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5175c-c826-4237-94d6-49adae5cca3f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:04.787350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5175c-c826-4237-94d6-49adae5cca3f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:04.822007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:04.822802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:04.823085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:04.823361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:04.845086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:04.852905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:04.853846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:04.854689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 365/722] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:04.859859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c573-48c8-4924-a88c-9f44c4b7dabe None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:05.000512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c573-48c8-4924-a88c-9f44c4b7dabe None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:05.048331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:05.048922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.049206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.049488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.050246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:05.050557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:05.052566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:05.061730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 358/723] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:05.069288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:05.108669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:05.199104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce9c55-d8a3-4001-92eb-f420ae9dc726 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:05.245856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:05.246659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.246941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.247214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.263372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:05.264851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:05.286562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:05.287577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:05.288529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:05.288529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:05.288529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:05.288529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:05.288529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 366/724] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:05.294357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:05.393973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:05.445990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ba54a-144c-4f9f-a822-d068c7f0dcd5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:05.480340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059586a-3b08-437d-8b21-491b0bc10a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:05.481002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059586a-3b08-437d-8b21-491b0bc10a52 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.481282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059586a-3b08-437d-8b21-491b0bc10a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.481556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059586a-3b08-437d-8b21-491b0bc10a52 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.482287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059586a-3b08-437d-8b21-491b0bc10a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:05.482594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059586a-3b08-437d-8b21-491b0bc10a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:05.483986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059586a-3b08-437d-8b21-491b0bc10a52 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:05.497607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 359/725] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:05.505103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:05.640138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175d49-999a-469c-b6d7-fde7b6233d5c None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.640505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175d49-999a-469c-b6d7-fde7b6233d5c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.640701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175d49-999a-469c-b6d7-fde7b6233d5c None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.642326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 360/726] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:05.647593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc5482-2462-41d8-9df7-65a657daf36d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.647851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc5482-2462-41d8-9df7-65a657daf36d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.648128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc5482-2462-41d8-9df7-65a657daf36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.668266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-66cc5482-2462-41d8-9df7-65a657daf36d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:05.669462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-66cc5482-2462-41d8-9df7-65a657daf36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:05.705394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:05.753787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0d444-2810-45b9-a7cc-20e58f389b24 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:05.771611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 361/727] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:05.798361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc5482-2462-41d8-9df7-65a657daf36d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:05.845114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:05.845929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.846215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.846493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.868201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:05.873234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:05.874697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:05.900576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:05.901551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:11:05.903518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:05.908765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 367/728] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:05.915005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:05.948729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc5482-2462-41d8-9df7-65a657daf36d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.021529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.022334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.022614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.022884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.041213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:06.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:06.050722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:06.051597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 362/729] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:11:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:06.060756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.090854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf235f-9659-4138-a06a-b6543e622607 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.125954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.126990 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.127478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.127989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.129066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:06.129602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:06.131801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:06.148220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:06.152859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 368/730] 10.222.0.62 () {56 vars in 1135 bytes} [Fri May 27 03:11:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:06.158166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.232919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c764955-2a9b-4836-b29b-197d9ac1b478 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.264196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:06.272420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a1527-c611-46d5-a9e4-94c67984f566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.273357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a1527-c611-46d5-a9e4-94c67984f566 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.273935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a1527-c611-46d5-a9e4-94c67984f566 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.274414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a1527-c611-46d5-a9e4-94c67984f566 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.275520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a1527-c611-46d5-a9e4-94c67984f566 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:06.276073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a1527-c611-46d5-a9e4-94c67984f566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:06.277822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a1527-c611-46d5-a9e4-94c67984f566 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:06.284578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303962e-998e-46c9-9eb7-b210f306fd3a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.288557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 363/731] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:06.290357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500801dd-e15b-47a3-8d82-df387f080aea None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.290863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500801dd-e15b-47a3-8d82-df387f080aea None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.291357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500801dd-e15b-47a3-8d82-df387f080aea None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.293524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 364/732] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:06.294444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500801dd-e15b-47a3-8d82-df387f080aea None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.335623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfef77-84cd-41be-a89b-545d7657b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.336312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfef77-84cd-41be-a89b-545d7657b439 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.336596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfef77-84cd-41be-a89b-545d7657b439 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.336871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfef77-84cd-41be-a89b-545d7657b439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.416108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500801dd-e15b-47a3-8d82-df387f080aea None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.416789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:06.419117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfef77-84cd-41be-a89b-545d7657b439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:06.421031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfef77-84cd-41be-a89b-545d7657b439 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:06.459062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2c9bb-5aa0-44ff-8f9c-97bd62754673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.460134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2c9bb-5aa0-44ff-8f9c-97bd62754673 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.460134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2c9bb-5aa0-44ff-8f9c-97bd62754673 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.460436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2c9bb-5aa0-44ff-8f9c-97bd62754673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.470834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2c9bb-5aa0-44ff-8f9c-97bd62754673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:06.472313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2c9bb-5aa0-44ff-8f9c-97bd62754673 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6ae2c9bb-5aa0-44ff-8f9c-97bd62754673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:06.490893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:06.491859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 365/733] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d926b8-bf56-4ab6-8d07-c5113e27d0de None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d926b8-bf56-4ab6-8d07-c5113e27d0de None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.497600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d926b8-bf56-4ab6-8d07-c5113e27d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.512598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-33d926b8-bf56-4ab6-8d07-c5113e27d0de None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:06.513770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-33d926b8-bf56-4ab6-8d07-c5113e27d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:06.531166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:06.536935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 369/734] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:11:06.542545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc2ad6-1ab3-4e47-8065-087bbc0bf1bf None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.542847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc2ad6-1ab3-4e47-8065-087bbc0bf1bf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.543130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc2ad6-1ab3-4e47-8065-087bbc0bf1bf None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.547856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 370/735] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:06.547856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc2ad6-1ab3-4e47-8065-087bbc0bf1bf None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.589408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 366/736] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:06.590221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d926b8-bf56-4ab6-8d07-c5113e27d0de None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.657988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:06.663443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc2ad6-1ab3-4e47-8065-087bbc0bf1bf None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.694851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d926b8-bf56-4ab6-8d07-c5113e27d0de None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.698750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628f696-0aef-4e8d-928b-16f5a9216ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.699458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f696-0aef-4e8d-928b-16f5a9216ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.699747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f696-0aef-4e8d-928b-16f5a9216ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.700054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f696-0aef-4e8d-928b-16f5a9216ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.700816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f696-0aef-4e8d-928b-16f5a9216ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:06.701137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f696-0aef-4e8d-928b-16f5a9216ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:06.702690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f696-0aef-4e8d-928b-16f5a9216ab9 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:06.722220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 371/737] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:06.722220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305b7f0-a250-4071-912b-fe17688f5d47 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.722220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305b7f0-a250-4071-912b-fe17688f5d47 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.722220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305b7f0-a250-4071-912b-fe17688f5d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.737681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0305b7f0-a250-4071-912b-fe17688f5d47 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:06.738884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0305b7f0-a250-4071-912b-fe17688f5d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:06.739978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.740895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.741181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.741476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.752778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:06.754233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:06.772556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:06.773484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:06.779030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:06.779030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:06.779030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:06.779030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:06.779030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 367/738] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:06.779030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.781621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:06.817336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 372/739] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:06.818455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305b7f0-a250-4071-912b-fe17688f5d47 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.912820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd955e-11e7-4ba2-a95a-8d739f446dd5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.915237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:06.927734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305b7f0-a250-4071-912b-fe17688f5d47 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:06.947651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.948404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.948682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.948952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.959449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:06.961771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:06.962580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:06.962868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:06.963151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:06.967140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:06.968215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 368/740] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:06.969079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:06.971406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:06.973480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:06.987754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:06.988721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 373/741] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:06.990680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.033044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.071006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282312d-c9e1-48ce-a017-5161e723c7fb None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.088834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87b28-83e5-4e40-82c9-484fcfbe72f4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.103576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd9719-93fb-4580-bba3-ecd91e9d07c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.104278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd9719-93fb-4580-bba3-ecd91e9d07c0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.104559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd9719-93fb-4580-bba3-ecd91e9d07c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.104833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd9719-93fb-4580-bba3-ecd91e9d07c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.105553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd9719-93fb-4580-bba3-ecd91e9d07c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:07.105860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd9719-93fb-4580-bba3-ecd91e9d07c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.107216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd9719-93fb-4580-bba3-ecd91e9d07c0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.123294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 369/742] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:07.123294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b812c0-7a2e-492c-9c80-e2d37a0d981e None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.123294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b812c0-7a2e-492c-9c80-e2d37a0d981e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.123294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b812c0-7a2e-492c-9c80-e2d37a0d981e None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.123650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.123650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.123936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.124214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.124947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.126306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.126623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 370/743] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:07.127380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b812c0-7a2e-492c-9c80-e2d37a0d981e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.140103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 374/744] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:06 2022] POST /identity/v3/services => generated 240 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:07.143315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.157490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.250249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b812c0-7a2e-492c-9c80-e2d37a0d981e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.262934 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad4265-6582-4e28-a460-f2a45251a3c1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.274143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.290963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.291656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.291979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.292261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.293039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:07.293354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.294751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.303499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1312974-8df7-45d6-a51a-98a7c2cc8c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.304130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 371/745] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:07.304688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1312974-8df7-45d6-a51a-98a7c2cc8c1d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.304987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.305350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1312974-8df7-45d6-a51a-98a7c2cc8c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.305633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1312974-8df7-45d6-a51a-98a7c2cc8c1d None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.306554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1312974-8df7-45d6-a51a-98a7c2cc8c1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:07.306871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1312974-8df7-45d6-a51a-98a7c2cc8c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.308377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1312974-8df7-45d6-a51a-98a7c2cc8c1d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.328887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 375/746] 10.222.0.62 () {56 vars in 1149 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:07.330400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ea9e3-79aa-4e85-bbd9-e99c203e91ba None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.330694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ea9e3-79aa-4e85-bbd9-e99c203e91ba None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.330973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ea9e3-79aa-4e85-bbd9-e99c203e91ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.348308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c11ea9e3-79aa-4e85-bbd9-e99c203e91ba None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:07.350086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c11ea9e3-79aa-4e85-bbd9-e99c203e91ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:07.424698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.429774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd1f0b-61a5-4629-be67-78feef7cbad0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.439846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 376/747] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:07 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:11:07.439846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ea9e3-79aa-4e85-bbd9-e99c203e91ba None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.472123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.473307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.473690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.473967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.481137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.482711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:07.498521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:07.499436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:07.500424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 372/748] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:07.500940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.546575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.550382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ea9e3-79aa-4e85-bbd9-e99c203e91ba None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.586484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.587266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.587546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.587841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.598290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.599752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.609399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c868ca-e595-4080-9470-fa75a3c623ea None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:07.617514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:07.618576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:07.619747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:07.619747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:07.619747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:07.619747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:07.619747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 377/749] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:07.620059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.653711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.654367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.654656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.654951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.674712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.730630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564852-eeec-4794-838c-4f8c83f6c1de None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.735499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.737105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.772284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.773723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.774012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.774293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.775031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.776425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.781149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.799867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 378/750] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:07 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:07.799867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.864044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 373/751] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:11:07.865599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.950317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495669a6-f61d-46a0-977e-39ce5e2456bf None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.958628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.985347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b4281-fabd-4870-8a7e-c68af07305f3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.993293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.994963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.995250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.995531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.996425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:07.996715 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.998156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.013378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 379/752] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:08.014038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.031038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.031483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.031769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.032077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.032814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:08.033128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.034699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.049567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 374/753] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:08.050411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.059538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.136832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500be9db-8a02-46a3-89da-b019feddb5da None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.178416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179ebe7-825d-422c-bff8-9e7b3d02fc15 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.180414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.181209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.181493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.181765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.200085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.201840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.218181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.218829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.219117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.219397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.220293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=4b633648662a4b51bad083fd4637f204) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:08.220607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.222095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.228991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 375/754] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/role_assignments?user.id=4b633648662a4b51bad083fd4637f204&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:08.229477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:08.230348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:08.231246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:08.231246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:08.231246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:08.231246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:08.231483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.238002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 380/755] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:08.239002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.259997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.321202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.365929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7922b-87c6-46db-8a71-896792047f59 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.406795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.408142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.408690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.409175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.413648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae8fb-c6cc-4310-af20-8f952afc2a0f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.419389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.421327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:08.439283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:08.440403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:08.441350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:08.448131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 376/756] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:08.449899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.453182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.455037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.455542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.456052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.457033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:08.457561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.459777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.478439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 381/757] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:08.484131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.562038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40554b73-e788-4baa-8156-ad820c592400 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.568244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.588099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.591908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52534997-81cc-4b08-b48d-eb86bffea3c5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.599768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.600465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.600744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.601018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.601865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:08.602172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.603615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.622568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 377/758] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:08.636187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.637365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.637863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.638359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.638730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.646113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.647859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:08.669187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:08.670149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:08.671043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:08.673555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 382/759] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:08.679695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.755234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf63b2b-4288-4498-837e-f4e5fa754d28 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.800013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.800823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.801102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.801376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.821838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.823339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:08.844471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:08.845410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:08.846335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:08.846335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:08.846335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:08.846335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:08.851696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa47b2-b76b-429f-ae4a-8b5bf584ecf4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.853537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 378/760] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:08.858559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.889272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.890333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.890822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.891350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.892432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:08.892968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.894609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.903114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.909582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 383/761] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:08.915440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.945630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:09.036882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312942f-fe25-455c-9423-5397fa241b33 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:09.076711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:09.077800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:09.078291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:09.078771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:09.079881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=1c908d4a33a041d8acc6ecad8c39ac6f) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:09.080421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:09.082729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:09.090739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 384/762] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/role_assignments?user.id=1c908d4a33a041d8acc6ecad8c39ac6f&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:09.126227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cb29c-8c7b-4cf8-b98e-a3b3843844f1 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:09.182517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:09.183201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:09.183546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:09.183849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:09.184611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:09.184924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:09.186315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:09.204180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 379/763] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:09.210136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:09.311223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:09.360798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028db27-bd19-483a-ba63-9b5b3db88f95 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:09.397917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:09.398634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:09.398918 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:09.399195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:09.406154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:09.407602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:09.421508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:09.422446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:09.423357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 385/764] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:09.428315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:09.562145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:09.569766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba98ee-c004-42e7-ab5e-5f0e7cdc1641 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:09.610993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37ac16-11d0-4a29-8cb6-3c8fda6dddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:09.611659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ac16-11d0-4a29-8cb6-3c8fda6dddbc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:09.611972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ac16-11d0-4a29-8cb6-3c8fda6dddbc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:09.612251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ac16-11d0-4a29-8cb6-3c8fda6dddbc None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:09.613051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ac16-11d0-4a29-8cb6-3c8fda6dddbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:09.613359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ac16-11d0-4a29-8cb6-3c8fda6dddbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:09.615325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ac16-11d0-4a29-8cb6-3c8fda6dddbc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:09.630918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 380/765] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:09.636762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:09.773244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685a244-b8bd-4f82-aa3c-2fd3bb257a66 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:09.806491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:09.825479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261aab-0665-45ec-947a-3570514d1727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:09.826377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261aab-0665-45ec-947a-3570514d1727 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:09.826658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261aab-0665-45ec-947a-3570514d1727 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:09.826931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261aab-0665-45ec-947a-3570514d1727 None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/edc938bed535461e974daf01bb99e9a6/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:09.829065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b261aab-0665-45ec-947a-3570514d1727 None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=edc938bed535461e974daf01bb99e9a6, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:09.830500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b261aab-0665-45ec-947a-3570514d1727 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:09.868993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 386/766] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:11:09 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/edc938bed535461e974daf01bb99e9a6/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:09.884966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c84f2-f393-4a77-aab6-877ebe672ef0 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:09.885309 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c84f2-f393-4a77-aab6-877ebe672ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:09.885544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c84f2-f393-4a77-aab6-877ebe672ef0 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:09.887294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 381/767] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:09.894967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54be5-cd07-49ac-87ea-962fcd72d87e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:09.894967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54be5-cd07-49ac-87ea-962fcd72d87e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:09.894967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54be5-cd07-49ac-87ea-962fcd72d87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:09.912720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-66c54be5-cd07-49ac-87ea-962fcd72d87e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:09.914301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-66c54be5-cd07-49ac-87ea-962fcd72d87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:09.992531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 387/768] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:10.016770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c84f2-f393-4a77-aab6-877ebe672ef0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:10.128641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c84f2-f393-4a77-aab6-877ebe672ef0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:10.165077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:10.166227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.166715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.167194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] PATH_INFO: `/v3/services/network` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.168107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:10.169676 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:10.181816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:10.182811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:10.183642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:10.186516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 382/769] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:10 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:10.193153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c54be5-cd07-49ac-87ea-962fcd72d87e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:10.293817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c54be5-cd07-49ac-87ea-962fcd72d87e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:10.310062 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:10.335348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:10.336439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.336929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.337580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.338926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] RBAC: Adding query filter params (name=network) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:10.339460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:10.341159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:10.346833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 388/770] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:10.352785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:10.376765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:10.464727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30a039-b6c5-42bf-8eb8-92302a847261 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:10.500168 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:10.501240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.501726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.502207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.503153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:10.503700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:10.505971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:10.512484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 383/771] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:10.518671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:10.621981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7272-bc37-43b2-8f7b-910c11740d9e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:10.658187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dba84-550c-40d7-841f-134e11953929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:10.659251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dba84-550c-40d7-841f-134e11953929 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.659741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dba84-550c-40d7-841f-134e11953929 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.660244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dba84-550c-40d7-841f-134e11953929 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.661907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dba84-550c-40d7-841f-134e11953929 None admin] RBAC: Adding query filter params (interface=public, service_id=dd8fcf632ab148f68550a45536bfe534, region_id=RegionOne) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:10.662430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dba84-550c-40d7-841f-134e11953929 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:10.664041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dba84-550c-40d7-841f-134e11953929 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:10.672353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 389/772] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:11:10 2022] GET /identity/v3/endpoints?service_id=dd8fcf632ab148f68550a45536bfe534&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:10.678094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:10.707186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87722ee-cdc3-4182-9889-b5fbf4f03c64 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.708368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87722ee-cdc3-4182-9889-b5fbf4f03c64 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.708870 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87722ee-cdc3-4182-9889-b5fbf4f03c64 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.710652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 390/773] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:10.716483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ddfb3-c1d2-4dba-83cc-c56d9d871ac2 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.716974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ddfb3-c1d2-4dba-83cc-c56d9d871ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.717436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ddfb3-c1d2-4dba-83cc-c56d9d871ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.746576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4b0ddfb3-c1d2-4dba-83cc-c56d9d871ac2 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:10.748084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4b0ddfb3-c1d2-4dba-83cc-c56d9d871ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:10.790113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:10.823744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 391/774] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:10.852225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:10.859853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ddfb3-c1d2-4dba-83cc-c56d9d871ac2 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:10.882295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f8d7c-d37c-4993-99c1-c9e7c26b12e6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:10.972338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ddfb3-c1d2-4dba-83cc-c56d9d871ac2 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:10.992303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:10.992986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.993271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.993549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.994253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:10.995640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.011695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 384/775] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:10 2022] GET /identity/v3/services => generated 772 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:11.035265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eba144-84eb-4d91-9b93-40f659c143a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.036226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eba144-84eb-4d91-9b93-40f659c143a0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.036523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eba144-84eb-4d91-9b93-40f659c143a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.036802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eba144-84eb-4d91-9b93-40f659c143a0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.037499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eba144-84eb-4d91-9b93-40f659c143a0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.038951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eba144-84eb-4d91-9b93-40f659c143a0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-12eba144-84eb-4d91-9b93-40f659c143a0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:11.066883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:11.068498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:11.069963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 392/776] 10.222.0.62 () {56 vars in 1129 bytes} [Fri May 27 03:11:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:11.075104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.184571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d7a3d-731c-4717-9d8e-3b7f8ab948b5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.209213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e405c4-2a89-40c6-8961-50f4eecf64c1 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.209727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e405c4-2a89-40c6-8961-50f4eecf64c1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.209803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e405c4-2a89-40c6-8961-50f4eecf64c1 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.216348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 393/777] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:11.217979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3f0c1-580e-46e7-a038-a5fde9dbeca9 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.218281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3f0c1-580e-46e7-a038-a5fde9dbeca9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.219010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3f0c1-580e-46e7-a038-a5fde9dbeca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.235469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.236115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.236398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.236676 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.237402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:11.237711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.239446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-86a3f0c1-580e-46e7-a038-a5fde9dbeca9 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:11.239632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.240682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-86a3f0c1-580e-46e7-a038-a5fde9dbeca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:11.253414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 385/778] 10.222.0.62 () {56 vars in 1124 bytes} [Fri May 27 03:11:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:11.259565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.285967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:11.304210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:11.360698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 394/779] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:11.386055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3f0c1-580e-46e7-a038-a5fde9dbeca9 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.405469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24528568-d784-4d9a-91b7-7d320e031726 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.464170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.465067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.465354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.466736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.467472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:11.467791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.469208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.480758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 386/780] 10.222.0.62 () {56 vars in 1124 bytes} [Fri May 27 03:11:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:11.486328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.543463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3f0c1-580e-46e7-a038-a5fde9dbeca9 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.623329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:11.633102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.633912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.634201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.634480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.650688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449402cd-8ea3-4d67-95a1-261e5b247940 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.656374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:11.664800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:11.665720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:11.666559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:11.678608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 395/781] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:11.685487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.692732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.693410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.693700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.693984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.694918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] RBAC: Adding query filter params (interface=public, service_id=6a4e019b42c84615bc391ccdcfccc25e, region_id=RegionOne) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:11.695241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.696776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.705451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 387/782] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:11:11 2022] GET /identity/v3/endpoints?service_id=6a4e019b42c84615bc391ccdcfccc25e&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:11.713466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.727767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:11.835212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d6913-a1b9-4be9-be4d-97f80d6e698b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.861536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ce685-a809-4b27-b7c8-05545286fe86 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.871305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66205e-cc51-4467-a6c7-bc9973ae85ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.872207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66205e-cc51-4467-a6c7-bc9973ae85ca None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.872643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66205e-cc51-4467-a6c7-bc9973ae85ca None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.873097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66205e-cc51-4467-a6c7-bc9973ae85ca None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.874047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66205e-cc51-4467-a6c7-bc9973ae85ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:11.874535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66205e-cc51-4467-a6c7-bc9973ae85ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.876183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66205e-cc51-4467-a6c7-bc9973ae85ca None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.885274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 396/783] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:11.886774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d958c21-02e9-4ffa-af5d-741302c11e05 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.887301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d958c21-02e9-4ffa-af5d-741302c11e05 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.887798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d958c21-02e9-4ffa-af5d-741302c11e05 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.893845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 397/784] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:11.894871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d958c21-02e9-4ffa-af5d-741302c11e05 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.905951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209a3f8-86e1-4431-ab9a-5dcc51c571d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.906887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209a3f8-86e1-4431-ab9a-5dcc51c571d7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.907317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209a3f8-86e1-4431-ab9a-5dcc51c571d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.908208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209a3f8-86e1-4431-ab9a-5dcc51c571d7 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.909061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209a3f8-86e1-4431-ab9a-5dcc51c571d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.911405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209a3f8-86e1-4431-ab9a-5dcc51c571d7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.925598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 388/785] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:11 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:11.928448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845858d-f259-42a7-9ce3-28a4a4ec6880 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.928896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845858d-f259-42a7-9ce3-28a4a4ec6880 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.929589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845858d-f259-42a7-9ce3-28a4a4ec6880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.946417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:11.968034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4845858d-f259-42a7-9ce3-28a4a4ec6880 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:11.969248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4845858d-f259-42a7-9ce3-28a4a4ec6880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:12.055721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:12.087722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d958c21-02e9-4ffa-af5d-741302c11e05 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.112055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 389/786] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:12.145432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.146245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.146533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.146829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.166764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:12.168160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845858d-f259-42a7-9ce3-28a4a4ec6880 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.168632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:12.191835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:12.192850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:12.193749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:12.193749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:12.193749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:12.193749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:12.193749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 398/787] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:12.201231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.285529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845858d-f259-42a7-9ce3-28a4a4ec6880 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.329539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.330763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.331303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.331782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.339414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da444e-a878-4479-a9c7-daafc8adfb25 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.340884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:12.348954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:12.358978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:12.359956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:11:12.360850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:12.372193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 390/788] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:12.373366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.385113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.385924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.386243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.386524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.387267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:12.387723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:12.390514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:12.409459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:12.421612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 399/789] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:12.427327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.536199 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358b2be-b571-4057-8516-0a948ab56abd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.569201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d1273-57fb-46d8-9d31-7f40e93757cf None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.585913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5aa1f8-e5f2-4026-b79f-0052c38de565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.586836 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5aa1f8-e5f2-4026-b79f-0052c38de565 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.587122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5aa1f8-e5f2-4026-b79f-0052c38de565 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.587398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5aa1f8-e5f2-4026-b79f-0052c38de565 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.589089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5aa1f8-e5f2-4026-b79f-0052c38de565 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:12.589403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5aa1f8-e5f2-4026-b79f-0052c38de565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:12.590855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5aa1f8-e5f2-4026-b79f-0052c38de565 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:12.603460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 391/790] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:12.605165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3761-a106-440e-8cb9-b96c45d17bf8 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.605509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3761-a106-440e-8cb9-b96c45d17bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.605790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3761-a106-440e-8cb9-b96c45d17bf8 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.606922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b7e9e-9552-46f7-a172-b4f646992e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.607423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 392/791] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:12.607794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b7e9e-9552-46f7-a172-b4f646992e79 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.608117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b7e9e-9552-46f7-a172-b4f646992e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.608543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b7e9e-9552-46f7-a172-b4f646992e79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.610320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3761-a106-440e-8cb9-b96c45d17bf8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.620654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b7e9e-9552-46f7-a172-b4f646992e79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:12.622294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b7e9e-9552-46f7-a172-b4f646992e79 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-503b7e9e-9552-46f7-a172-b4f646992e79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:12.637968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:12.639994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 400/792] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8527-46d8-4ecc-8e59-88309751284f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.642247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8527-46d8-4ecc-8e59-88309751284f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.643514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8527-46d8-4ecc-8e59-88309751284f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.661397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7f5d8527-46d8-4ecc-8e59-88309751284f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:12.662717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7f5d8527-46d8-4ecc-8e59-88309751284f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:12.689500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:12.723210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:12.745000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 401/793] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:12.745514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3761-a106-440e-8cb9-b96c45d17bf8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.745924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d8527-46d8-4ecc-8e59-88309751284f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.786715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.787515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.787805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.788091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:12.800589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:12.818364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:12.820616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:12.822718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:12.822718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:12.822718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:12.822718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:12.822718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 393/794] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:12.822718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.867724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d8527-46d8-4ecc-8e59-88309751284f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.906189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.906973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.907266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.907659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.908565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:12.909051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:12.911321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:12.926952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a24101-3653-4b32-bc2d-dfebdfeb3a38 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:12.931388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 402/795] 10.222.0.62 () {56 vars in 1135 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:12.931866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:12.941207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:12.968915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:12.969722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:12.970011 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:12.970293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:12.980968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:12.988646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:12.989892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:12.990844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:12.994203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:12.995002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 394/796] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:12.995900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.055705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03af3-4dbb-4f3a-8aa8-b22b0688ee27 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.095089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.096171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.096639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.097119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.098155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:13.098626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.100301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.100457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6a5c-ffa2-434a-9233-59fb88b6b8a8 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.113853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 403/797] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:13.115096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.136519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.137136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.137426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.137708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.149480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.150942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:13.168317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:13.169274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:13.170178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:13.170178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:13.170178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:13.170178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:13.174223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 395/798] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:13.174949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.198027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:13.233361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350a78-3ca0-4cfa-a1c8-3f0cafebeeb8 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.241579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:13.274776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.275936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.276542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.277045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.278125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:13.278676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.280617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.283143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165b976-dfaf-4bce-9fe2-73e3e71ad589 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.291152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 404/799] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:13.292278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.336600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.339340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.339783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.340229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.350565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.352043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:13.373763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:13.374801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:13.375788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 396/800] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:13.376415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.440586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa638ac-fa74-451e-9eaf-48030c28cb7f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.473092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:13.480361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.481191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.481573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.481995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.482827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:13.483239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.484729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.497869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 405/801] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:13.498854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.499944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5019e-67c5-429c-932c-a5958fd1f138 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.529073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:13.555840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.555840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.555840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.555840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.569886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.572362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:13.592346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:13.593451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:13.594487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:13.594487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:13.594487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:13.594487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:13.594487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 397/802] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:13.598854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.605205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1bf2f-9227-4bb3-9b51-4486e001d483 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.640975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c561-a31a-45e0-bc7e-3f4ac468e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.641650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c561-a31a-45e0-bc7e-3f4ac468e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.641939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c561-a31a-45e0-bc7e-3f4ac468e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.642229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c561-a31a-45e0-bc7e-3f4ac468e9c3 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.643075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c561-a31a-45e0-bc7e-3f4ac468e9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:13.643390 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c561-a31a-45e0-bc7e-3f4ac468e9c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.644846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c561-a31a-45e0-bc7e-3f4ac468e9c3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.662703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 406/803] 10.222.0.62 () {56 vars in 1149 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:13.663971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5b5c4-dc7e-4861-b2d2-a517c64190f0 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.664264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5b5c4-dc7e-4861-b2d2-a517c64190f0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.664546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5b5c4-dc7e-4861-b2d2-a517c64190f0 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.667852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 407/804] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:13.667852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5b5c4-dc7e-4861-b2d2-a517c64190f0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.713531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b248-9c6d-4692-b205-0eb91d9bc676 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.747184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:13.754265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.755215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.755509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.755973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:13.756083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.764397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.765853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.772630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5b5c4-dc7e-4861-b2d2-a517c64190f0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:13.780188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:13.781250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:13.782283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 398/805] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:13.783945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.812279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380a21-4c3c-4b22-a6c9-5f2c927722a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.812987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380a21-4c3c-4b22-a6c9-5f2c927722a5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.813306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380a21-4c3c-4b22-a6c9-5f2c927722a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.813588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380a21-4c3c-4b22-a6c9-5f2c927722a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.814324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380a21-4c3c-4b22-a6c9-5f2c927722a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:13.814640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380a21-4c3c-4b22-a6c9-5f2c927722a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.816040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380a21-4c3c-4b22-a6c9-5f2c927722a5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.828183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 408/806] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:13.829418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a9e3-5a14-4cd8-ad39-c47aca2da4e8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.829711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a9e3-5a14-4cd8-ad39-c47aca2da4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.829995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a9e3-5a14-4cd8-ad39-c47aca2da4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.846347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-1bb4a9e3-5a14-4cd8-ad39-c47aca2da4e8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:13.847542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1bb4a9e3-5a14-4cd8-ad39-c47aca2da4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:13.899745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890ba7-084c-40df-bbbc-d15dda080b3b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.918608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 409/807] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:13.918608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4a9e3-5a14-4cd8-ad39-c47aca2da4e8 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.954756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.955978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.956414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.956854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.968317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.970062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:13.990179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:13.991177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:13.992128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:13.992128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:13.992128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:13.992128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:13.992128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 399/808] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:13.992885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.008025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.033278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4a9e3-5a14-4cd8-ad39-c47aca2da4e8 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.043360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.070545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.071202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.071488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.071769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.072589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:14.072903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.074323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.086969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 410/809] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:14.087850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.112046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61582bf4-8b9c-4530-9d9f-119757ed6f93 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.149993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.151198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.151640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.151958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.160251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.162202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:14.176674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:14.177687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:14.178695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 400/810] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:14.179308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.202083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03895566-bbf9-4615-9e0e-46b4b9bfe54b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.241897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.242740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.243036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.243311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] PATH_INFO: `/v3/services/network` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.244035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.245427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.248695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:14.259075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:14.260388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:14.261267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 411/811] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:14.262073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.293295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6f564-506a-4e75-b8f4-d3e7a4e49f0e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.296494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.329729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.330263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.330536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.330816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.331547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:14.331893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.334430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.350420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 401/812] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:14.350420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.375272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c4493-8aed-4056-80ff-6b7f06930473 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.412518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.413407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.413695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.413979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/4b633648662a4b51bad083fd4637f204/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.415952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=4b633648662a4b51bad083fd4637f204, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.417536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.430528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 412/813] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:11:14 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/4b633648662a4b51bad083fd4637f204/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:14.431344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.468773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e98e-115f-46f9-8695-9aa896b7aee8 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.498866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.509471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.510408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.510548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.510730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.511575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:14.511934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.513411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.534509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 402/814] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:14.535366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.555740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656b1a4-ca8c-43a7-9e6e-355a05f3606c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.563362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.595447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e942f4-fd95-4bde-a2eb-160816bc5939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.596195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e942f4-fd95-4bde-a2eb-160816bc5939 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.596489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e942f4-fd95-4bde-a2eb-160816bc5939 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.596777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e942f4-fd95-4bde-a2eb-160816bc5939 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.597519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e942f4-fd95-4bde-a2eb-160816bc5939 None admin] RBAC: Adding query filter params (name=network) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:14.597839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e942f4-fd95-4bde-a2eb-160816bc5939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.599293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e942f4-fd95-4bde-a2eb-160816bc5939 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.609440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 413/815] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:14.609440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2660c3-2947-4129-a9df-5e5a9995d520 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.609440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2660c3-2947-4129-a9df-5e5a9995d520 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.609440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2660c3-2947-4129-a9df-5e5a9995d520 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.609440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 414/816] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:14.609440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2660c3-2947-4129-a9df-5e5a9995d520 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.659844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08dc65-47a8-4e81-967f-c2ffc915cce0 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.696274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.697165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.697446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.697718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.705299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.706801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:14.722468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:14.723455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:14.724407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 403/817] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:14.725591 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.730749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2660c3-2947-4129-a9df-5e5a9995d520 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.762121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.773599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.774254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.774538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.774816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.811390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.832120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5d55c-9194-491b-9da1-3f89961f0ebd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.845426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.846858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.869188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dae886-e2ec-494b-88ef-a9cb1f11efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.870289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae886-e2ec-494b-88ef-a9cb1f11efd2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.870664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae886-e2ec-494b-88ef-a9cb1f11efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.870947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae886-e2ec-494b-88ef-a9cb1f11efd2 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.871664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dae886-e2ec-494b-88ef-a9cb1f11efd2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:14.872002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dae886-e2ec-494b-88ef-a9cb1f11efd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.873369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dae886-e2ec-494b-88ef-a9cb1f11efd2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.879135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 404/818] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:14.880901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a81e8-3fc8-410b-a25f-55ba6485cb6d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.881196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a81e8-3fc8-410b-a25f-55ba6485cb6d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.881478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a81e8-3fc8-410b-a25f-55ba6485cb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.898274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-592a81e8-3fc8-410b-a25f-55ba6485cb6d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:14.899451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-592a81e8-3fc8-410b-a25f-55ba6485cb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:14.955699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 415/819] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:14.960863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.975649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 405/820] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:14.976318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a81e8-3fc8-410b-a25f-55ba6485cb6d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.998195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.055698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.078509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0892b-9b13-4140-ac04-2a6afa3ecdb2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.079936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a81e8-3fc8-410b-a25f-55ba6485cb6d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.114378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.115046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.115331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.115611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.115970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.116440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:15.116629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.116786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.116933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.117180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.117923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.118171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.119305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.136777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 416/821] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:15.136777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.144794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 406/822] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:15.145918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.262704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.285999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44f294-61b4-4986-9931-d9d6e58d721d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.291212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cd05c-682f-4fca-b6aa-0cf6c4e81afc None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.317429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.329138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.330360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.331274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.331998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.332920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.334569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.341882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.342687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.342971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.343296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:15.350311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:15.351679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:11:15.352550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:15.355440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.355661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 407/823] 10.222.0.62 () {56 vars in 1129 bytes} [Fri May 27 03:11:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:15.356814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.356995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:15.376389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:15.377322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:15.378193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:15.378193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:15.378193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:15.378193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:15.380140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 417/824] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:15.381061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.497208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10369c49-8962-4235-9151-7df8bcee7836 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.499524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385085-8d96-4cb5-9a4f-64b9e92bf9d0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.533271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7088c-da39-427c-9667-767043e315fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.533952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7088c-da39-427c-9667-767043e315fd None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.534233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7088c-da39-427c-9667-767043e315fd None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.534510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7088c-da39-427c-9667-767043e315fd None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.535241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7088c-da39-427c-9667-767043e315fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:15.536319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7088c-da39-427c-9667-767043e315fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.538211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7088c-da39-427c-9667-767043e315fd None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.538336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.539123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.539523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.539685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/1c908d4a33a041d8acc6ecad8c39ac6f/roles/cf8ecb4f65d74cccbc07ab4e466a76ad` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.540821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.544973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 418/825] 10.222.0.62 () {56 vars in 1124 bytes} [Fri May 27 03:11:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:15.546214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7088c-da39-427c-9667-767043e315fd None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.560260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=1c908d4a33a041d8acc6ecad8c39ac6f, role_id=cf8ecb4f65d74cccbc07ab4e466a76ad)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.561713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.583941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 408/826] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:11:15 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/1c908d4a33a041d8acc6ecad8c39ac6f/roles/cf8ecb4f65d74cccbc07ab4e466a76ad => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:15.583941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.631877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.699385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7088c-da39-427c-9667-767043e315fd None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.701319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58237174-a2e1-4965-b164-0c7f85d6416a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.740041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.740781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.740998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.741271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.741633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.742009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:15.742318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.742417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.742784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.743136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.743691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.744063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:15.744437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.745892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.751862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 409/827] 10.222.0.62 () {56 vars in 1124 bytes} [Fri May 27 03:11:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:15.757165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 419/828] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:15.760931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.762866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:15.833385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.877916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e7c9f-fccb-4e6f-8657-2d4dca84b031 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.878497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cef181-3726-465f-a067-e1ab10c78fb3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:15.887495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:15.922177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.923219 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.923689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.924186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.925387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:15.926088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:15.926374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:15.926652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:15.927692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.929568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.938404 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:15.940025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:15.952290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 410/829] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:15.955630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:15.956662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:15.958244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 420/830] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:15.968020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:16.096031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac994-91ee-4795-8e7f-9878739ab75a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:16.134679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78301b42-add2-4941-a785-3466a2bc6b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:16.135271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78301b42-add2-4941-a785-3466a2bc6b03 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.135556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78301b42-add2-4941-a785-3466a2bc6b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.135851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78301b42-add2-4941-a785-3466a2bc6b03 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.136671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78301b42-add2-4941-a785-3466a2bc6b03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:16.136986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78301b42-add2-4941-a785-3466a2bc6b03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:16.138486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78301b42-add2-4941-a785-3466a2bc6b03 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:16.152646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 411/831] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:16.158326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:16.262265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31928e9d-2a39-4722-9cf5-a6ddbef94ad8 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:16.282782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:16.296832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861b429-1e37-413c-b53f-68b1e2f8b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:16.297491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b429-1e37-413c-b53f-68b1e2f8b14d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.297774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b429-1e37-413c-b53f-68b1e2f8b14d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.298051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b429-1e37-413c-b53f-68b1e2f8b14d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.298890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861b429-1e37-413c-b53f-68b1e2f8b14d None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=edc938bed535461e974daf01bb99e9a6) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:16.299202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861b429-1e37-413c-b53f-68b1e2f8b14d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:16.300665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861b429-1e37-413c-b53f-68b1e2f8b14d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:16.307766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 421/832] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:16 2022] GET /identity/v3/role_assignments?user.id=edc938bed535461e974daf01bb99e9a6&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:16.690648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baca436-95c9-4ada-8d36-7ef83d56af6b None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.691173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baca436-95c9-4ada-8d36-7ef83d56af6b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.691173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baca436-95c9-4ada-8d36-7ef83d56af6b None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.693006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 412/833] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:16.698232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e7e23-ae34-4bfb-9bc0-20c3b59b7dad None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.698625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e7e23-ae34-4bfb-9bc0-20c3b59b7dad None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.698796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e7e23-ae34-4bfb-9bc0-20c3b59b7dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.716797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-177e7e23-ae34-4bfb-9bc0-20c3b59b7dad None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:16.717993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-177e7e23-ae34-4bfb-9bc0-20c3b59b7dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:16.786501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 422/834] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:16.810325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baca436-95c9-4ada-8d36-7ef83d56af6b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:16.830635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:16.920217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baca436-95c9-4ada-8d36-7ef83d56af6b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:16.955385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:16.956150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.956434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.956713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.967377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:16.975499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:16.976479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:16.977343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 413/835] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:16.982905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e7e23-ae34-4bfb-9bc0-20c3b59b7dad None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.079903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e7e23-ae34-4bfb-9bc0-20c3b59b7dad None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.094393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:17.114801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.115491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.115779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.116090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.116879 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:17.117194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.118681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.128855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 423/836] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:17.134513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.218107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:17.240244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc26ebe-a00d-4e0a-88a8-ebdf0c96347f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.275301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.276136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.276419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.276693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.288118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.289621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:17.306698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:17.307607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:17.308559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:17.308559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:17.308559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:17.308559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:17.308559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 414/837] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:17.313750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.414706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c543bde-8fed-41f5-868f-4ea40df08c8e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.448193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.448859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.449193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.449477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.450204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:17.450512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.451905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.464838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 424/838] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:17.470441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.576088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2782d33-182e-4c92-9fa1-bd653d4f4f40 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.605500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:17.611786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.612615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.612910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.613173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.621686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.623147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:17.638428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:17.639388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:17.640359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 415/839] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:17.646123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.681067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:17.752156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f876-70f8-45fa-b1f6-a43f99dceeaa None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.789181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808db87-03bc-4395-8d14-875b300e6bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.789882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808db87-03bc-4395-8d14-875b300e6bdf None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.790165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808db87-03bc-4395-8d14-875b300e6bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.790442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808db87-03bc-4395-8d14-875b300e6bdf None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.791312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808db87-03bc-4395-8d14-875b300e6bdf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:17.791626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808db87-03bc-4395-8d14-875b300e6bdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.793157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808db87-03bc-4395-8d14-875b300e6bdf None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.813885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 425/840] 10.222.0.62 () {56 vars in 1135 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:17.819896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b8d85-d8e2-4e30-91d5-afe2cf1b4607 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.977877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde4cc-104c-49e0-8b3c-fa88a8d69237 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.978164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde4cc-104c-49e0-8b3c-fa88a8d69237 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.978447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde4cc-104c-49e0-8b3c-fa88a8d69237 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.981179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618471-b628-4834-8b96-ef806516841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.981981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618471-b628-4834-8b96-ef806516841b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.982420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618471-b628-4834-8b96-ef806516841b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.982840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618471-b628-4834-8b96-ef806516841b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.983506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 426/841] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:17.984647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef13f-5e46-4cc2-bf62-d58c9dd9c6d5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.984932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef13f-5e46-4cc2-bf62-d58c9dd9c6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.985209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef13f-5e46-4cc2-bf62-d58c9dd9c6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.011240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-403ef13f-5e46-4cc2-bf62-d58c9dd9c6d5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:18.013323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-403ef13f-5e46-4cc2-bf62-d58c9dd9c6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:18.053989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.069379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db618471-b628-4834-8b96-ef806516841b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.070859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db618471-b628-4834-8b96-ef806516841b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.108141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 427/842] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:18.133151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef13f-5e46-4cc2-bf62-d58c9dd9c6d5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.167074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.205244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 416/843] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:18.210348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618471-b628-4834-8b96-ef806516841b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.261165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef13f-5e46-4cc2-bf62-d58c9dd9c6d5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.297651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07c905-05b7-4eb6-8022-d2409174b2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.298528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07c905-05b7-4eb6-8022-d2409174b2af None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.298816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07c905-05b7-4eb6-8022-d2409174b2af None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.299215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07c905-05b7-4eb6-8022-d2409174b2af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.310825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07c905-05b7-4eb6-8022-d2409174b2af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:18.311469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-ee07c905-05b7-4eb6-8022-d2409174b2af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:18.319367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:18.320415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:18.321256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 428/844] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:18.322492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051e295-fab7-46d5-bbd6-9ecc7e30d93b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.322492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051e295-fab7-46d5-bbd6-9ecc7e30d93b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.322492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051e295-fab7-46d5-bbd6-9ecc7e30d93b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.327873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 429/845] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:18.327873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051e295-fab7-46d5-bbd6-9ecc7e30d93b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.330398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618471-b628-4834-8b96-ef806516841b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.367584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf197a6-cd4c-4035-8741-c25faba7956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.368407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf197a6-cd4c-4035-8741-c25faba7956c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.368740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf197a6-cd4c-4035-8741-c25faba7956c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.369019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf197a6-cd4c-4035-8741-c25faba7956c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.379573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf197a6-cd4c-4035-8741-c25faba7956c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.381046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf197a6-cd4c-4035-8741-c25faba7956c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-0cf197a6-cd4c-4035-8741-c25faba7956c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:18.399312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:18.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 417/846] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03e62-aa70-4caa-b4d9-b9c2b4ed6fe0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03e62-aa70-4caa-b4d9-b9c2b4ed6fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.405928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03e62-aa70-4caa-b4d9-b9c2b4ed6fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.420311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-29b03e62-aa70-4caa-b4d9-b9c2b4ed6fe0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:18.421476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-29b03e62-aa70-4caa-b4d9-b9c2b4ed6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:18.427372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.447898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051e295-fab7-46d5-bbd6-9ecc7e30d93b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.486981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.488078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.489101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.489378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.490177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:18.490450 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.491840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.506316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 430/847] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:18.506316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.506641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 418/848] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:18.507269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b03e62-aa70-4caa-b4d9-b9c2b4ed6fe0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.585268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.617590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de8ccd-d736-4c3e-8eac-d37899a04870 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.620651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b03e62-aa70-4caa-b4d9-b9c2b4ed6fe0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.652835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.653500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.653785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.654063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.654785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:18.655097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.655903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.656494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.656695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.656976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.657251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.672657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 431/849] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:18.672657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.674334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.675879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.680084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:18.693806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:18.694748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:18.695660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:18.695660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:18.695660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:18.695660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:18.695660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 419/850] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:18.696686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.774004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59230fd1-3629-4d29-9cc4-64595e38600e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.793258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7777e6-95c9-4c75-868d-6ec605565ee4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.806460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe9363-9b36-46e6-b728-629cdd6bd24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.807252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe9363-9b36-46e6-b728-629cdd6bd24e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.807532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe9363-9b36-46e6-b728-629cdd6bd24e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.807812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe9363-9b36-46e6-b728-629cdd6bd24e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.808488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe9363-9b36-46e6-b728-629cdd6bd24e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.809844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe9363-9b36-46e6-b728-629cdd6bd24e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.816126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-2abe9363-9b36-46e6-b728-629cdd6bd24e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:18.821330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:18.822416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:18.823249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 432/851] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:18.824079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c14a5-4d1c-4b7c-906a-34ef48d985bc None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.824418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c14a5-4d1c-4b7c-906a-34ef48d985bc None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.824658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c14a5-4d1c-4b7c-906a-34ef48d985bc None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.825771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 433/852] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:18.826448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c14a5-4d1c-4b7c-906a-34ef48d985bc None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.833747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.834549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.834834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.835112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.843379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.844791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:18.858722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:18.859641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:18.860563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:18.864087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 420/853] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:18.864941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:18.896093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:18.933976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c14a5-4d1c-4b7c-906a-34ef48d985bc None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.962572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fe5b3-faba-4e6c-baa7-fbb08164fee5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:18.967384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825e96f-3997-4412-ace0-f077ed8aa4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.968074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825e96f-3997-4412-ace0-f077ed8aa4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.968358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825e96f-3997-4412-ace0-f077ed8aa4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.968633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825e96f-3997-4412-ace0-f077ed8aa4b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.969349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825e96f-3997-4412-ace0-f077ed8aa4b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:18.969661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825e96f-3997-4412-ace0-f077ed8aa4b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.971010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825e96f-3997-4412-ace0-f077ed8aa4b9 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:18.982252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 434/854] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:18.983410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11743e-0a4a-4915-9e21-18e8ffb75917 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.983697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11743e-0a4a-4915-9e21-18e8ffb75917 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.984007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11743e-0a4a-4915-9e21-18e8ffb75917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.995408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:18.996103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:18.996393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:18.996673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:18.997390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:18.997700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:18.999065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.000024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fb11743e-0a4a-4915-9e21-18e8ffb75917 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:19.001182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fb11743e-0a4a-4915-9e21-18e8ffb75917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:19.003758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 421/855] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.004949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.053115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.067126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 435/856] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:19.067987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11743e-0a4a-4915-9e21-18e8ffb75917 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.104752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d32c5-a2ae-4ee4-9041-64f4ebcd481d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.125632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.137477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.138138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.138420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.138694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.139488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:19.139800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.141257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.154230 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 422/857] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.155012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.166635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11743e-0a4a-4915-9e21-18e8ffb75917 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.200102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.200907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.201193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.201472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.208844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.210268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:19.225230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:19.226382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 436/858] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:19.227381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.255891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf173b7-c763-42df-8ae0-29307f53d015 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.264841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.290570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.291230 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.291514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.291790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.292532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:19.292841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.294206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.298547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 423/859] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.310409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.331508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5ef5-6856-4121-befe-0e8f2dcb85e3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.356763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.374554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.375457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.375886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.376166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.376837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.378382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:19.392873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:19.393997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:19.400206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 437/860] 10.222.0.62 () {56 vars in 1132 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:19.401233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.431758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586063f-c74a-48fd-821d-940b7ed3dbc7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.470534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.471609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.472140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.472415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.473310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=4b633648662a4b51bad083fd4637f204) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:19.473621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.475183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.481978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 424/861] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/role_assignments?user.id=4b633648662a4b51bad083fd4637f204&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.483335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.529164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9de3b3-8c01-4722-8178-542230c18ad0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.544643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.581416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.582489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.582938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.583375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.584423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:19.584909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.586562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.609303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 438/862] 10.222.0.62 () {56 vars in 1149 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.615851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.624940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726af36-076d-44d9-bfdf-c9093b4c934b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.667146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.670915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.671595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.671911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.672191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.672920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:19.673228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.674609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.680485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 425/863] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.686550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:19.737848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af516e2-cc94-4f90-a39b-aaa8303783f5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.779562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.780258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.780542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.780830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.806611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.812074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5eee2-2e86-4dba-92a7-2ef6ee3cfecd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:19.855394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:19.856591 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.856879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.857162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.857895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:19.858210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.859646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.865276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 426/864] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.881992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:19.883935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:19.933604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:19.998222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 439/865] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:20.003701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.112969 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782293a-508d-4ae0-8de2-cbc4ec4533f2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.148300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.149093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.149375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.149653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.160818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.162276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:20.180218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:20.181162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:20.182075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:20.182075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:20.182075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:20.182075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:20.185941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 427/866] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:20.187291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.287242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f706461-8470-49f2-bfcd-6d92e5fec72f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.323663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.324369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.324652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.324929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.325662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:20.325971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.327333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.344810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 440/867] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:20.346138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.365979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:20.474022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41c42-8e52-4418-afaf-3d15cfe6ac75 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.511431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.512266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.512549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.512826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.519724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.521247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:20.535640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:20.537686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:20.539782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:20.543842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 428/868] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:20.544052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.560176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:20.651503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0da1-741f-4871-8ed8-7d1e9720c8b4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.685350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f342b2-ca74-478f-8552-02d4e85ebe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.686017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f342b2-ca74-478f-8552-02d4e85ebe8c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.686303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f342b2-ca74-478f-8552-02d4e85ebe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.686580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f342b2-ca74-478f-8552-02d4e85ebe8c None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.687378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f342b2-ca74-478f-8552-02d4e85ebe8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:20.687692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f342b2-ca74-478f-8552-02d4e85ebe8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.689164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f342b2-ca74-478f-8552-02d4e85ebe8c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.706328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 441/869] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:20.707706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.793249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:20.816692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2195064-fd0f-46af-89a0-03a8af4fa1c7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.852366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2217f5-14f8-477c-803e-4f7c83f99458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.853023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2217f5-14f8-477c-803e-4f7c83f99458 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.853307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2217f5-14f8-477c-803e-4f7c83f99458 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.853585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2217f5-14f8-477c-803e-4f7c83f99458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.854423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2217f5-14f8-477c-803e-4f7c83f99458 None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=1c908d4a33a041d8acc6ecad8c39ac6f) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:20.854735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2217f5-14f8-477c-803e-4f7c83f99458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.856183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2217f5-14f8-477c-803e-4f7c83f99458 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.867983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 429/870] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/role_assignments?user.id=1c908d4a33a041d8acc6ecad8c39ac6f&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:21.605739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eceb52-e12c-4c8d-80b6-a05afaee85a3 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:21.606267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eceb52-e12c-4c8d-80b6-a05afaee85a3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:21.606346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eceb52-e12c-4c8d-80b6-a05afaee85a3 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:21.607888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 442/871] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:21.613796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b098-29d1-4f0c-9edc-236c1d67b9bc None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:21.614092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b098-29d1-4f0c-9edc-236c1d67b9bc None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:21.614380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b098-29d1-4f0c-9edc-236c1d67b9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:21.632207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9748b098-29d1-4f0c-9edc-236c1d67b9bc None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:21.633377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9748b098-29d1-4f0c-9edc-236c1d67b9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:21.706504 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 430/872] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:21.729382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eceb52-e12c-4c8d-80b6-a05afaee85a3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:21.797750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:21.838873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eceb52-e12c-4c8d-80b6-a05afaee85a3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:21.867079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17c47-3a1c-4fec-a5e8-046f48492cb1 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:21.867669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17c47-3a1c-4fec-a5e8-046f48492cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:21.867669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17c47-3a1c-4fec-a5e8-046f48492cb1 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:21.869182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 431/873] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:21.873941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:21.874607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:21.874736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b728d-f97d-4590-87c2-8ddc6ba6325e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:21.874911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:21.875009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b728d-f97d-4590-87c2-8ddc6ba6325e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:21.875157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:21.875250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b728d-f97d-4590-87c2-8ddc6ba6325e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:21.875898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:21.877257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:21.894101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fe4b728d-f97d-4590-87c2-8ddc6ba6325e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:21.895423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fe4b728d-f97d-4590-87c2-8ddc6ba6325e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:21.896753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 443/874] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:21 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:21.983742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:21.998901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 432/875] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:22.021352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:22.135061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05490fbf-8c9b-44cd-a1b6-14e80001e920 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:22.159245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:22.173285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:22.174545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:22.175038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:22.175562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:22.176505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:22.178194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:22.192293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:22.193379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:22.194224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:22.197299 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 444/876] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:22.203500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b728d-f97d-4590-87c2-8ddc6ba6325e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:22.306509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b728d-f97d-4590-87c2-8ddc6ba6325e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:22.342526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7235a5b-fdef-4f72-91c0-21ded78d368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:22.343191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235a5b-fdef-4f72-91c0-21ded78d368d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:22.343471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235a5b-fdef-4f72-91c0-21ded78d368d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:22.343745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235a5b-fdef-4f72-91c0-21ded78d368d None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:22.344492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7235a5b-fdef-4f72-91c0-21ded78d368d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:22.344798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7235a5b-fdef-4f72-91c0-21ded78d368d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:22.346165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7235a5b-fdef-4f72-91c0-21ded78d368d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:22.351328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 433/877] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:22.357262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:22.380636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e51fa-83b8-4cd5-9eb1-acfa65588172 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:22.380946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e51fa-83b8-4cd5-9eb1-acfa65588172 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:22.381202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e51fa-83b8-4cd5-9eb1-acfa65588172 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:22.383505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 434/878] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:22.390054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b005-9ed5-48f2-bfd0-bce8b4fbba19 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:22.390294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b005-9ed5-48f2-bfd0-bce8b4fbba19 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:22.390583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b005-9ed5-48f2-bfd0-bce8b4fbba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:22.412952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b405b005-9ed5-48f2-bfd0-bce8b4fbba19 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:22.414427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b405b005-9ed5-48f2-bfd0-bce8b4fbba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:22.470199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1266f85-a739-4f38-b635-c728554c65af None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:22.499769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:22.509863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651310a-377a-4d2f-8e3c-2fc4475a372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:22.510681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651310a-377a-4d2f-8e3c-2fc4475a372b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:22.511066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651310a-377a-4d2f-8e3c-2fc4475a372b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:22.511417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651310a-377a-4d2f-8e3c-2fc4475a372b None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:22.512260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651310a-377a-4d2f-8e3c-2fc4475a372b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:22.512643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651310a-377a-4d2f-8e3c-2fc4475a372b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:22.514097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651310a-377a-4d2f-8e3c-2fc4475a372b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:22.522121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 435/879] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:22.525762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 445/880] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:22.540865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405b005-9ed5-48f2-bfd0-bce8b4fbba19 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:22.665434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405b005-9ed5-48f2-bfd0-bce8b4fbba19 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:22.703375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5d17-84b5-456b-8cb2-071b6d0408d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:22.704087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5d17-84b5-456b-8cb2-071b6d0408d3 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:22.704370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5d17-84b5-456b-8cb2-071b6d0408d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:22.704647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5d17-84b5-456b-8cb2-071b6d0408d3 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:22.705400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f5d17-84b5-456b-8cb2-071b6d0408d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:22.706828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f5d17-84b5-456b-8cb2-071b6d0408d3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:22.724595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 436/881] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:22 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:23.186737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab33204-dbd7-4239-a46d-0fc89cca1ac0 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.187231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab33204-dbd7-4239-a46d-0fc89cca1ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.187231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab33204-dbd7-4239-a46d-0fc89cca1ac0 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.188880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 446/882] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:23.194348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa97f5-6ab3-4d9f-9c38-41bb598306c6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.195140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa97f5-6ab3-4d9f-9c38-41bb598306c6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.195568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa97f5-6ab3-4d9f-9c38-41bb598306c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.213609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f3fa97f5-6ab3-4d9f-9c38-41bb598306c6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:23.215013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f3fa97f5-6ab3-4d9f-9c38-41bb598306c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:23.262164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:23.295901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 437/883] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:23.313890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab33204-dbd7-4239-a46d-0fc89cca1ac0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.366778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:23.419712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab33204-dbd7-4239-a46d-0fc89cca1ac0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.453610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.454348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.454632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.454907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.466463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:23.474620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:23.475561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:23.476441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 447/884] 10.222.0.62 () {56 vars in 1120 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:23.481765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa97f5-6ab3-4d9f-9c38-41bb598306c6 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.584039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa97f5-6ab3-4d9f-9c38-41bb598306c6 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.618841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.619570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.619878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.620160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.620917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:23.621225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:23.622700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:23.631690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 438/885] 10.222.0.62 () {56 vars in 1115 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:23.637009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.744382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbc529-814c-461c-a030-452cb6e2eaa1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.756323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:23.781642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.782829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.783322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.783793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.794625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:23.796357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:23.814593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:23.815541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:23.818604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:23.818604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:23.818604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:23.818604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:23.820678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 448/886] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:23.831850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.873163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:23.951617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3a033-b506-427f-97e0-b9d60da88562 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.985758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.986426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.986710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.986988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.987718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:23.988579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:23.990067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.002792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 439/887] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:24.008923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.117478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c4aef-f639-4615-a5b4-0062d2836a89 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.153351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdbf91-4411-46bd-ac91-7ff77d8ef57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.154163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdbf91-4411-46bd-ac91-7ff77d8ef57c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.154448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdbf91-4411-46bd-ac91-7ff77d8ef57c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.154724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdbf91-4411-46bd-ac91-7ff77d8ef57c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.162999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdbf91-4411-46bd-ac91-7ff77d8ef57c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.164463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdbf91-4411-46bd-ac91-7ff77d8ef57c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-80fdbf91-4411-46bd-ac91-7ff77d8ef57c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:24.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:24.180780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:24.182034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 449/888] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:24.188813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.296891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c72475-7999-48c8-9d60-1b615e27b5d3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.299121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.320380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26df2e-5188-44ee-99bc-58c5fb7d8ec8 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.320921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26df2e-5188-44ee-99bc-58c5fb7d8ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.320921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26df2e-5188-44ee-99bc-58c5fb7d8ec8 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.322856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 450/889] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:24.327863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a864b3-4be6-45e2-b9c6-9c69b97e47ca None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.328158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a864b3-4be6-45e2-b9c6-9c69b97e47ca None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.328857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a864b3-4be6-45e2-b9c6-9c69b97e47ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.344732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.345742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.346227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.346704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.347953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-82a864b3-4be6-45e2-b9c6-9c69b97e47ca None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:24.348596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:24.349111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.349216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-82a864b3-4be6-45e2-b9c6-9c69b97e47ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:24.350837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.370431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.374269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 440/890] 10.222.0.62 () {56 vars in 1149 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:24.376275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.447842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 451/891] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:24.466721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a864b3-4be6-45e2-b9c6-9c69b97e47ca None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.493838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e874f3d-fa9f-4f7f-b535-69cad173bc47 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.533128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.534303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.535398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.535919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.562656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.590316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a864b3-4be6-45e2-b9c6-9c69b97e47ca None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.623447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.625448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.638863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.640005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.640294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.640574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.641242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.642800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:24.660683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:24.661757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:24.662611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 452/892] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:24.670010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.691584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.755974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 441/893] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:24.758754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.792487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e30a4a-39ff-4175-abe0-1f64c2a46fa0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.829325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc917-635a-4188-89df-9a66726aa45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.830343 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc917-635a-4188-89df-9a66726aa45e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.830628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc917-635a-4188-89df-9a66726aa45e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.830903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc917-635a-4188-89df-9a66726aa45e None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.832338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdc917-635a-4188-89df-9a66726aa45e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:24.832649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdc917-635a-4188-89df-9a66726aa45e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.834026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdc917-635a-4188-89df-9a66726aa45e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.834520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.839847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 453/894] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:24.840643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad32e0-ebb7-4a2b-8c5c-bb809728b69f None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.840928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad32e0-ebb7-4a2b-8c5c-bb809728b69f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.841200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad32e0-ebb7-4a2b-8c5c-bb809728b69f None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.842596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 454/895] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:24.847788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad32e0-ebb7-4a2b-8c5c-bb809728b69f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.895449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340b59b-6ff4-4855-8b60-132d22a3cb0e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.937344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e86153-bdb0-4615-a883-97a2d9b82893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.938146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86153-bdb0-4615-a883-97a2d9b82893 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.938465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86153-bdb0-4615-a883-97a2d9b82893 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.938885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86153-bdb0-4615-a883-97a2d9b82893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.942348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.950251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e86153-bdb0-4615-a883-97a2d9b82893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.952145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e86153-bdb0-4615-a883-97a2d9b82893 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.965285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad32e0-ebb7-4a2b-8c5c-bb809728b69f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-78e86153-bdb0-4615-a883-97a2d9b82893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:24.970592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:24.971508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:24.972489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:24.972489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:24.972489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:24.972489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:24.972489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 442/896] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:24.973368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bcbff-4d8b-47da-8dc5-4d60ac198f05 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.973651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bcbff-4d8b-47da-8dc5-4d60ac198f05 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.973932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bcbff-4d8b-47da-8dc5-4d60ac198f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.990264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-dc6bcbff-4d8b-47da-8dc5-4d60ac198f05 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:24.991462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dc6bcbff-4d8b-47da-8dc5-4d60ac198f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:25.008939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.010327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.010803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.011264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.012706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:25.013276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.015389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.021600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 455/897] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:25.026462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.065752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 443/898] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:25.067027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bcbff-4d8b-47da-8dc5-4d60ac198f05 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.110467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:25.135489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd037349-eded-4318-bb29-44ac880f3a56 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.175202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bcbff-4d8b-47da-8dc5-4d60ac198f05 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.176366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.177117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.177410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.177676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.178410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:25.178722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.180114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.192353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 456/899] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:25.193113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.196353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:25.215589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424781eb-9abb-4c82-8801-d48ba8cea5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.216273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424781eb-9abb-4c82-8801-d48ba8cea5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.216557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424781eb-9abb-4c82-8801-d48ba8cea5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.216830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424781eb-9abb-4c82-8801-d48ba8cea5d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.217616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424781eb-9abb-4c82-8801-d48ba8cea5d7 None admin] RBAC: Adding query filter params (interface=public, service_id=f2337441a8114df2ad38257ea8ab32f4, region_id=RegionOne) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:25.217929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424781eb-9abb-4c82-8801-d48ba8cea5d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.219297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424781eb-9abb-4c82-8801-d48ba8cea5d7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.224247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 444/900] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/endpoints?service_id=f2337441a8114df2ad38257ea8ab32f4&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:25.225536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20a70-0e53-4edd-bcbc-1aa5d110c897 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.225830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20a70-0e53-4edd-bcbc-1aa5d110c897 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.226129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20a70-0e53-4edd-bcbc-1aa5d110c897 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.231612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 445/901] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:25.231612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20a70-0e53-4edd-bcbc-1aa5d110c897 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.297280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285d9c6-a4ce-4010-9e21-c0a392fbdf49 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.330070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20a70-0e53-4edd-bcbc-1aa5d110c897 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.331741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:25.331883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.332508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.332840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.333065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.333791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.335166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.348388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 457/902] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:25 2022] POST /identity/v3/services => generated 241 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:25.349429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.375971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f587f63-aaa1-4bc8-97cb-254e502c570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.376703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f587f63-aaa1-4bc8-97cb-254e502c570b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.376982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f587f63-aaa1-4bc8-97cb-254e502c570b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.377255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f587f63-aaa1-4bc8-97cb-254e502c570b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.385587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f587f63-aaa1-4bc8-97cb-254e502c570b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.387078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f587f63-aaa1-4bc8-97cb-254e502c570b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2f587f63-aaa1-4bc8-97cb-254e502c570b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:25.401755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:25.402937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:25.403876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 446/903] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:25.404620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f024-0fc5-49b9-a32c-6b0f655a5831 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.404966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f024-0fc5-49b9-a32c-6b0f655a5831 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.405246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f024-0fc5-49b9-a32c-6b0f655a5831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.424097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e8b4f024-0fc5-49b9-a32c-6b0f655a5831 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:25.425844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e8b4f024-0fc5-49b9-a32c-6b0f655a5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:25.428664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:25.480773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dbee-4155-411f-8ec5-dcab0516e545 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.500381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 447/904] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:25.501494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f024-0fc5-49b9-a32c-6b0f655a5831 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.530312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.530535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.530919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.531526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.532268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.533753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.541503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 458/905] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:11:25.542297 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.617636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:25.642738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f024-0fc5-49b9-a32c-6b0f655a5831 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.668469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec61c3b-e1ca-4631-8e93-212c33bc75a5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.692942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.693771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.694155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.694532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.695449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:25.695888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.697493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.708758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:25.713491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 448/906] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:25.719301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.725488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.726383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.726667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.726942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.727685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.729168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:25.742548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:25.743551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:25.744426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 459/907] 10.222.0.62 () {56 vars in 1132 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:25.749887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.841819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275414b-2efc-4308-a3c0-9bb82a9bc8dc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.876075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dc8df-8514-44fe-96b4-54ef6b5ba439 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:25.878569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.879229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.879511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.879785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.880658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] RBAC: Adding query filter params (role.id=f88a7c60dc0b4604be9ca139688c9ae7, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=1c908d4a33a041d8acc6ecad8c39ac6f) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:25.880969 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.882408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.888870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 449/908] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/role_assignments?user.id=1c908d4a33a041d8acc6ecad8c39ac6f&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=f88a7c60dc0b4604be9ca139688c9ae7 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:25.916014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:25.916622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:25.916906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:25.920125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:25.920125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:25.920125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:25.920125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:25.927332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 460/909] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:25.934218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:25.949598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:26.049378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40191d5e-c6a9-48b1-ab3f-1a8f41a07df6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:26.086658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:26.087388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:26.088072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:26.088349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:26.089067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:26.089379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:26.090750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:26.096268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 450/910] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:26.102028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:26.159670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:26.206163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cebcb-5867-4ecc-94d2-aa2a5c8dd934 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:26.241606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812841c-ad57-4b81-8577-589d19986aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:26.242268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812841c-ad57-4b81-8577-589d19986aef None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:26.242556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812841c-ad57-4b81-8577-589d19986aef None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:26.242870 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812841c-ad57-4b81-8577-589d19986aef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:26.243674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812841c-ad57-4b81-8577-589d19986aef None admin] RBAC: Adding query filter params (interface=public, service_id=e690cb3fc00b4663af79cbf5987c3a90, region_id=RegionOne) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:26.244012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812841c-ad57-4b81-8577-589d19986aef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:26.245378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812841c-ad57-4b81-8577-589d19986aef None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:26.251107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 461/911] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:11:26 2022] GET /identity/v3/endpoints?service_id=e690cb3fc00b4663af79cbf5987c3a90&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:26.257129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:26.341308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:26.365268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408f1f-3e20-4a29-b286-0df2f4c91afc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:26.399959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fedb27-d317-49dc-a313-9545a5cc1417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:26.400632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fedb27-d317-49dc-a313-9545a5cc1417 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:26.400914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fedb27-d317-49dc-a313-9545a5cc1417 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:26.401191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fedb27-d317-49dc-a313-9545a5cc1417 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:26.401879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fedb27-d317-49dc-a313-9545a5cc1417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:26.403254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fedb27-d317-49dc-a313-9545a5cc1417 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:26.410355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 451/912] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:26 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:11:27.680990 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50e47-a694-40c0-8913-9b41da7f58f0 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.681552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50e47-a694-40c0-8913-9b41da7f58f0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.681552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50e47-a694-40c0-8913-9b41da7f58f0 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.683131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 462/913] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:27.688572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d25f-b11f-4f67-baba-ee4b108b26be None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.689013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d25f-b11f-4f67-baba-ee4b108b26be None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.689197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d25f-b11f-4f67-baba-ee4b108b26be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.709232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fb09d25f-b11f-4f67-baba-ee4b108b26be None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:27.710582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fb09d25f-b11f-4f67-baba-ee4b108b26be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:27.784829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 452/914] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:27.807539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50e47-a694-40c0-8913-9b41da7f58f0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:27.905459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a162-c4d6-42a4-8e21-220bcf5ff460 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.905781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a162-c4d6-42a4-8e21-220bcf5ff460 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.906010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a162-c4d6-42a4-8e21-220bcf5ff460 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.907521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 453/915] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:27.913239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d514-cd3a-4f06-a0c1-423003d151f1 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.913780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d514-cd3a-4f06-a0c1-423003d151f1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.914275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d514-cd3a-4f06-a0c1-423003d151f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.918719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50e47-a694-40c0-8913-9b41da7f58f0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:27.942758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:27.944806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8b59d514-cd3a-4f06-a0c1-423003d151f1 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:27.946438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8b59d514-cd3a-4f06-a0c1-423003d151f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:27.964536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:27.965368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.965654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.965933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.966879 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:27.968550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:27.985607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:27.986683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:27.987617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 463/916] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:27.995504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.029303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.043855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 454/917] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:28.061187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59d514-cd3a-4f06-a0c1-423003d151f1 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.108087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ebb76-654d-43af-b00f-7eb3a102e43d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.144956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b05e-8fdc-443f-bd5e-6621d4168fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.146182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b05e-8fdc-443f-bd5e-6621d4168fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.146466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b05e-8fdc-443f-bd5e-6621d4168fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.146743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b05e-8fdc-443f-bd5e-6621d4168fb0 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.147463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b05e-8fdc-443f-bd5e-6621d4168fb0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:28.147775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b05e-8fdc-443f-bd5e-6621d4168fb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.149174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b05e-8fdc-443f-bd5e-6621d4168fb0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.159196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 464/918] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:28.161289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409c895-6c9f-48db-b6ee-034e45fb3b2a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.161578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409c895-6c9f-48db-b6ee-034e45fb3b2a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.161863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409c895-6c9f-48db-b6ee-034e45fb3b2a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.164239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 465/919] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:28.164239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409c895-6c9f-48db-b6ee-034e45fb3b2a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.176795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59d514-cd3a-4f06-a0c1-423003d151f1 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.210501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b2d2a-4972-4f6b-92ca-f269e5269356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.211213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b2d2a-4972-4f6b-92ca-f269e5269356 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.211497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b2d2a-4972-4f6b-92ca-f269e5269356 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.211766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b2d2a-4972-4f6b-92ca-f269e5269356 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.212717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b2d2a-4972-4f6b-92ca-f269e5269356 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.214074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b2d2a-4972-4f6b-92ca-f269e5269356 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.221630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-962b2d2a-4972-4f6b-92ca-f269e5269356 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:28.227829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:28.228865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:11:28.229718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:28.231960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 455/920] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:28.234503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be2f22-a0e7-423a-a10c-ea929a934776 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.235216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be2f22-a0e7-423a-a10c-ea929a934776 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.235672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be2f22-a0e7-423a-a10c-ea929a934776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.257634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-78be2f22-a0e7-423a-a10c-ea929a934776 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:28.259109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-78be2f22-a0e7-423a-a10c-ea929a934776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:28.283685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409c895-6c9f-48db-b6ee-034e45fb3b2a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.318275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.323350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.324061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.324348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.324630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.325345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:28.325657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.327031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.327538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 456/921] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:28.337075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 466/922] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:28.337233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be2f22-a0e7-423a-a10c-ea929a934776 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.338129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.448408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be2f22-a0e7-423a-a10c-ea929a934776 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.452980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d762bb-dc8c-4d8b-968f-bf8768c2d908 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.468128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.484639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.485305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.485594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.485871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.486574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:28.486887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.488814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.491368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.492070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.492346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.492623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.493420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] RBAC: Adding query filter params (interface=public, service_id=554df699ac514b5aaf6b8f04c38fb304, region_id=RegionOne) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:28.493656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 457/923] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:28.493753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.495225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.496951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.500617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 467/924] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/endpoints?service_id=554df699ac514b5aaf6b8f04c38fb304&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:28.501440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.568333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.608820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ca300-8ee5-4d52-9b2c-cc5a1cce4d1c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.613233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24f09-c36f-4be7-9bd5-d5888e9e728d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.642750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.643408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.643760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.644100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.644808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:28.645118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.646475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.647916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab936b-7810-428f-b2a2-1ee14a93fa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.648730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab936b-7810-428f-b2a2-1ee14a93fa02 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.649020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab936b-7810-428f-b2a2-1ee14a93fa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.649398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab936b-7810-428f-b2a2-1ee14a93fa02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.651239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 468/925] 10.222.0.62 () {56 vars in 1114 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:28.652049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.660797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab936b-7810-428f-b2a2-1ee14a93fa02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-d2ab936b-7810-428f-b2a2-1ee14a93fa02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:28.669032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:28.670044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:28.670893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 458/926] 10.222.0.62 () {56 vars in 1120 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:28.671607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713e98-1afa-4a83-8b38-0ec35e48b97e None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.671715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713e98-1afa-4a83-8b38-0ec35e48b97e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.672019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713e98-1afa-4a83-8b38-0ec35e48b97e None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.673351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 459/927] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:28.674031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713e98-1afa-4a83-8b38-0ec35e48b97e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.697718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.757099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecacde-ecce-44f0-a839-116cec8d0090 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.774909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713e98-1afa-4a83-8b38-0ec35e48b97e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.789556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.790213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.790500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.790779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.791470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.791643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.792873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.798701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 469/928] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:11:28.799468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:28.809778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c95ba-d1b9-434b-8c2b-7a1088dcee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.810588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c95ba-d1b9-434b-8c2b-7a1088dcee3b None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.810879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c95ba-d1b9-434b-8c2b-7a1088dcee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.811161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c95ba-d1b9-434b-8c2b-7a1088dcee3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.811983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c95ba-d1b9-434b-8c2b-7a1088dcee3b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:28.813426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c95ba-d1b9-434b-8c2b-7a1088dcee3b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:28.843854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 460/929] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:28.843854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82c1c-f86a-454a-bf63-5c9c9045ba79 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.843854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82c1c-f86a-454a-bf63-5c9c9045ba79 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.843854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82c1c-f86a-454a-bf63-5c9c9045ba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.863614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-53c82c1c-f86a-454a-bf63-5c9c9045ba79 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:28.865893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-53c82c1c-f86a-454a-bf63-5c9c9045ba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:28.949925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815db13a-38c7-4d4c-9151-ec1c1e4fa12f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:28.953803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:28.982834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 461/930] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:28.996317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:28.997327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:28.997758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:28.998192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:28.999118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:28.999594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.001168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.006169 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82c1c-f86a-454a-bf63-5c9c9045ba79 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.012594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 470/931] 10.222.0.62 () {56 vars in 1115 bytes} [Fri May 27 03:11:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:29.018208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.127599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:29.131630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82c1c-f86a-454a-bf63-5c9c9045ba79 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.152691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2917b8-555d-4fb9-a5f4-c21939d7b8f5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.183611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.184444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.184722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.184994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.185650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.187022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.204651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:29.204866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:29.205744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:29.207136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 462/932] 10.222.0.62 () {56 vars in 1132 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:29.213908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.213908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.213908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.214180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.227252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.229123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:29.251245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:29.252247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:29.253183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:29.253183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:29.253183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:29.253183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:29.253183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 471/933] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:29.258114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.282342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:29.342336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27977d-2220-4331-9d04-4a9e25b704b6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.368697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24705-f96e-4281-9c7f-b4a6169f0429 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.381192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.382152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.382638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.383120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.384197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:29.384691 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.386248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.396170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 463/934] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:29.398208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.410676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.411485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.411940 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.412742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.413877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:29.414295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.416081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.430377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:29.439857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 472/935] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:29.442019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.511143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffe09-dcd4-4fce-8b49-56d914610df9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.547097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.547792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.548185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.548532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.549316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:29.549690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.550421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:29.551233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.557159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 464/936] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:29.562219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.580621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e9b75-7ff9-4e68-983c-98e04047274e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.623512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.624825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.625109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.625388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.635860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.637829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:29.655771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:29.656772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:29.657735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:29.660510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 473/937] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:29.665848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.679971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:29.689971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3c56b-bc19-460b-8892-580f8ac325a4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.730015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.731161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.731765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.732344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.733295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.734911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.757546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 465/938] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:11:29.771039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa932-a885-442c-861b-8e112c9a0833 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:29.821517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:29.822215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.822559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.822832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.823732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:29.824071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:29.825523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:29.827447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:29.859036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 474/939] 10.222.0.62 () {56 vars in 1149 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:29.860874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.978331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dcd1e-a194-4395-8862-0264cc1fc427 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:30.015108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:30.015934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:30.016223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:30.016504 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:30.028448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:30.029942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:30.049895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:30.050843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:30.051754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:30.051754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:30.051754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:30.051754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:30.052868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:30.057039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 466/940] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:30.057476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:30.171513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaec2a-fd9a-4380-95c4-0caca41cb1cf None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:30.208316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:30.208989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:30.209276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:30.209557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:30.210302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:30.210616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:30.212017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:30.228370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 475/941] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:30.230654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:30.334298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095f8ba-22ff-491c-b376-0e672d0df332 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:30.368605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:30.369409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:30.369696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:30.369978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:30.376717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:30.378698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:30.393078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:30.394546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:30.395448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:30.399605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 467/942] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:30.400973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:30.459801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:30.510587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3361f-075a-478d-8714-ef5469d01087 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:30.551237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89027179-7139-46ff-8216-efa637e6047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:30.552465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89027179-7139-46ff-8216-efa637e6047b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:30.552975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89027179-7139-46ff-8216-efa637e6047b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:30.553496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89027179-7139-46ff-8216-efa637e6047b None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:30.555111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89027179-7139-46ff-8216-efa637e6047b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:30.555505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89027179-7139-46ff-8216-efa637e6047b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:30.557439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89027179-7139-46ff-8216-efa637e6047b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:30.575937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 476/943] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:30.577629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:30.640267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:30.689843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21891bac-b162-4949-bd7b-d23f50364759 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:30.725924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cefc9-9eba-4cdd-bed2-96dd26796b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:30.726792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cefc9-9eba-4cdd-bed2-96dd26796b7f None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:30.727082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cefc9-9eba-4cdd-bed2-96dd26796b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:30.727360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cefc9-9eba-4cdd-bed2-96dd26796b7f None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/1c908d4a33a041d8acc6ecad8c39ac6f/roles/f88a7c60dc0b4604be9ca139688c9ae7` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:30.729664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cefc9-9eba-4cdd-bed2-96dd26796b7f None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=1c908d4a33a041d8acc6ecad8c39ac6f, role_id=f88a7c60dc0b4604be9ca139688c9ae7)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:30.731233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cefc9-9eba-4cdd-bed2-96dd26796b7f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:30.747384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 468/944] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:11:30 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/1c908d4a33a041d8acc6ecad8c39ac6f/roles/f88a7c60dc0b4604be9ca139688c9ae7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:31.121041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7867-2367-47f3-8543-fbc8c8f93482 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.121578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7867-2367-47f3-8543-fbc8c8f93482 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.121875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7867-2367-47f3-8543-fbc8c8f93482 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.123648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 477/945] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:31.129163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726a65e-491b-4643-97ad-af257de99426 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.129567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726a65e-491b-4643-97ad-af257de99426 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.129925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726a65e-491b-4643-97ad-af257de99426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.147088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4726a65e-491b-4643-97ad-af257de99426 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:31.148318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4726a65e-491b-4643-97ad-af257de99426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:31.217149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfe675-d6c5-42d7-adab-f2b550015c28 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.217475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfe675-d6c5-42d7-adab-f2b550015c28 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.217718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfe675-d6c5-42d7-adab-f2b550015c28 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.219300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 478/947] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:31.222184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 469/947] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:31.224719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b590e8-f931-497f-ae1e-f8d79bb259c0 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.225013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b590e8-f931-497f-ae1e-f8d79bb259c0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.225293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b590e8-f931-497f-ae1e-f8d79bb259c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.243661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726a65e-491b-4643-97ad-af257de99426 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:31.248069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-15b590e8-f931-497f-ae1e-f8d79bb259c0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:31.249303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-15b590e8-f931-497f-ae1e-f8d79bb259c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:31.320892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 479/948] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:31.324061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:31.344851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b590e8-f931-497f-ae1e-f8d79bb259c0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:31.361299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726a65e-491b-4643-97ad-af257de99426 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:31.396491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:31.406967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:31.408382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.409753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.410492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.411399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:31.413185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:31.430782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:31.431837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:31.432729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:31.436170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 470/949] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:31.439583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:31.454880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b590e8-f931-497f-ae1e-f8d79bb259c0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:31.491666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:31.492365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.492652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.492932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.493767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:31.495274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:31.496699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING keystone.server.flask.common [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 03:11:31.514687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.sql.core [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:11:31.514687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:11:31.514687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [parameters: {'id': 'b3b871e3e1e64f39a295ea2aab9ed2de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:11:31.514687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80273) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3b871e3e1e64f39a295ea2aab9ed2de.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3b871e3e1e64f39a295ea2aab9ed2de. May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application conn.query(q) May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:11:31.524067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result.read() May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 03:11:31.524961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application next(self.gen) May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application yield resource May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application next(self.gen) May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise self.value May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application session.commit() May 27 03:11:31.525927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.session.flush() May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application compat.raise_( May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:11:31.527491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:11:31.528912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application conn.query(q) May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result.read() May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:11:31.530806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [parameters: {'id': 'b3b871e3e1e64f39a295ea2aab9ed2de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:31.533237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:31.535842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3b871e3e1e64f39a295ea2aab9ed2de. May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 480/950] 10.222.0.62 () {60 vars in 1140 bytes} [Fri May 27 03:11:31 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:31.538081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:31.588835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b05a-6718-4986-9c34-cc8eb24c5ab4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:31.609017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:31.636962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:31.638058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.638546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.639020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.639977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:31.640488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:31.642065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:31.647468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:31.648535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 471/951] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:31.656100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ced43-b0b8-424f-b599-9f0b40d4cd05 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:31.656262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:31.702391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:31.703972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.704418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.704854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.715491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:31.718484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:31.737572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:31.738775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:31.739974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:31.745897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 481/952] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:31.747805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:31.773712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd61b8-3b80-4814-87e1-d2ee8edae90c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:31.820816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:31.822909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.823399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.823897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.824800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:31.825291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:31.826858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:31.835324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 472/953] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:31.840725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:31.857441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dc715-d983-4084-a09a-d51752ba94df None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:31.899534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecf2ea-db1b-41f7-b452-e90d92254114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:31.900265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecf2ea-db1b-41f7-b452-e90d92254114 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:31.900661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecf2ea-db1b-41f7-b452-e90d92254114 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:31.901015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecf2ea-db1b-41f7-b452-e90d92254114 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:31.901873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecf2ea-db1b-41f7-b452-e90d92254114 None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:31.902197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:31.902308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecf2ea-db1b-41f7-b452-e90d92254114 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:31.903844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecf2ea-db1b-41f7-b452-e90d92254114 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:31.935849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 482/954] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:31.982251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439edea-1982-4a4d-9bc0-7541843fbbef None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:32.020921 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c655231-03de-4b23-b961-36e9eef7eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:32.021607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c655231-03de-4b23-b961-36e9eef7eea4 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.021893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c655231-03de-4b23-b961-36e9eef7eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.022358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c655231-03de-4b23-b961-36e9eef7eea4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.023216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c655231-03de-4b23-b961-36e9eef7eea4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:32.024657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c655231-03de-4b23-b961-36e9eef7eea4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:32.052354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 473/955] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:11:32.083806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b20d-e212-4c8d-a6e1-6c23e526ee98 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.084157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b20d-e212-4c8d-a6e1-6c23e526ee98 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.084442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b20d-e212-4c8d-a6e1-6c23e526ee98 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.086289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 483/956] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:32.093860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89162af-6e5b-45c7-bcf7-3d43ee2f91f8 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.094179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89162af-6e5b-45c7-bcf7-3d43ee2f91f8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.094429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89162af-6e5b-45c7-bcf7-3d43ee2f91f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.114422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d89162af-6e5b-45c7-bcf7-3d43ee2f91f8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:32.115721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d89162af-6e5b-45c7-bcf7-3d43ee2f91f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:32.208052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 474/957] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:32.231561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7b20d-e212-4c8d-a6e1-6c23e526ee98 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:32.252601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:32.340724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7b20d-e212-4c8d-a6e1-6c23e526ee98 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:32.378521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:32.379658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.380199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.380671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.381711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:32.383450 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:32.396036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:32.397136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:32.398106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:32.401033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 484/958] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:32.407183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89162af-6e5b-45c7-bcf7-3d43ee2f91f8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:32.434265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:32.519007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89162af-6e5b-45c7-bcf7-3d43ee2f91f8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:32.556372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b297a-4c43-4816-9000-4657ec81f227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:32.557040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b297a-4c43-4816-9000-4657ec81f227 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.557320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b297a-4c43-4816-9000-4657ec81f227 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.557592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b297a-4c43-4816-9000-4657ec81f227 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.558306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b297a-4c43-4816-9000-4657ec81f227 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:32.558612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b297a-4c43-4816-9000-4657ec81f227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:32.560000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b297a-4c43-4816-9000-4657ec81f227 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:32.565264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 475/959] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:32.570867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:32.660099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:32.681096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b795-5cee-44fb-af1d-f07392422b26 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:32.718025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575b225-a474-40b4-af4f-26fd1f3e6e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:32.719060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575b225-a474-40b4-af4f-26fd1f3e6e56 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.719556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575b225-a474-40b4-af4f-26fd1f3e6e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.720094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575b225-a474-40b4-af4f-26fd1f3e6e56 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.721015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575b225-a474-40b4-af4f-26fd1f3e6e56 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:32.722058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575b225-a474-40b4-af4f-26fd1f3e6e56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:32.723675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575b225-a474-40b4-af4f-26fd1f3e6e56 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:32.729249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 485/960] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:33.040174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc157396-0a28-4915-b8b2-7350b4662105 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:33.040501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc157396-0a28-4915-b8b2-7350b4662105 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:33.040764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc157396-0a28-4915-b8b2-7350b4662105 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:33.042585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 476/961] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:33.047951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180570-0ac7-44fc-89ec-71b707a29380 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:33.048246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180570-0ac7-44fc-89ec-71b707a29380 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:33.048528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180570-0ac7-44fc-89ec-71b707a29380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:33.065640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-27180570-0ac7-44fc-89ec-71b707a29380 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:33.067124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-27180570-0ac7-44fc-89ec-71b707a29380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:33.137052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 486/962] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:33.161575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc157396-0a28-4915-b8b2-7350b4662105 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:33.260291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:33.272998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc157396-0a28-4915-b8b2-7350b4662105 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:33.308052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:33.308830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:33.309116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:33.309394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:33.320533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:33.329097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:33.330078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:33.331499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 477/963] 10.222.0.62 () {56 vars in 1120 bytes} [Fri May 27 03:11:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:33.337334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180570-0ac7-44fc-89ec-71b707a29380 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:33.420256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:33.442983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180570-0ac7-44fc-89ec-71b707a29380 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:33.478794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:33.479466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:33.479752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:33.480065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:33.480821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:33.481132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:33.482497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:33.491240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 487/964] 10.222.0.62 () {56 vars in 1115 bytes} [Fri May 27 03:11:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:33.496446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:33.602191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ce39-e678-42d7-a2e1-ce9b0fb1e510 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:33.637246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:33.638037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:33.638322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:33.638608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:33.649425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:33.650883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:33.659212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:33.668878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:33.670891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:33.673029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:33.673029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:33.673029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:33.673029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:33.675636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 478/965] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:33.676633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:33.787243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfaae5-420c-4712-8627-a6027197a50d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:33.822579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:33.823248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:33.823539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:33.823832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:33.824587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:33.824902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:33.826283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:33.834225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:33.842983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 488/966] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:33.850044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:33.970508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec09f5-0a17-41c8-a553-5d50def38d64 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:34.006085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:34.007259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.007774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.008292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.016365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:34.018016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:34.033831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:34.034818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:11:34.035778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:34.038482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 479/967] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:11:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:34.044566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:34.155273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5fe30-4cb5-4da0-bdd2-0c41c72b917f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:34.194607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5efbe-9c29-4a16-abce-f90f2e5bf754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:34.195785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5efbe-9c29-4a16-abce-f90f2e5bf754 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.196294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5efbe-9c29-4a16-abce-f90f2e5bf754 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.196822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5efbe-9c29-4a16-abce-f90f2e5bf754 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.197872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5efbe-9c29-4a16-abce-f90f2e5bf754 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:34.198385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5efbe-9c29-4a16-abce-f90f2e5bf754 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:34.200078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5efbe-9c29-4a16-abce-f90f2e5bf754 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:34.224344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 489/968] 10.222.0.62 () {56 vars in 1149 bytes} [Fri May 27 03:11:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:34.226629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:34.289554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:34.331118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e4cd-91fc-459a-b692-2f4d8167c15c None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.331433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e4cd-91fc-459a-b692-2f4d8167c15c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.331681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e4cd-91fc-459a-b692-2f4d8167c15c None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.333625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 490/969] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:34.334583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e20e-0a84-48ef-a722-caeab1db01f5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:34.341463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12373d-3d0e-44a4-99f3-447410535c5e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.342353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12373d-3d0e-44a4-99f3-447410535c5e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.342896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12373d-3d0e-44a4-99f3-447410535c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.363645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7a12373d-3d0e-44a4-99f3-447410535c5e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:34.365210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7a12373d-3d0e-44a4-99f3-447410535c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:34.373616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b020a-b21c-4309-a9bc-81680f29a9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:34.374708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b020a-b21c-4309-a9bc-81680f29a9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.374996 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b020a-b21c-4309-a9bc-81680f29a9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.375276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b020a-b21c-4309-a9bc-81680f29a9b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.422218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:34.452242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 491/970] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:34.464757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b020a-b21c-4309-a9bc-81680f29a9b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:34.466231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b020a-b21c-4309-a9bc-81680f29a9b0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:34.473675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12373d-3d0e-44a4-99f3-447410535c5e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:34.553700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:34.605549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 480/971] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:34 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:11:34.605850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12373d-3d0e-44a4-99f3-447410535c5e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:34.607442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0c577-56d3-4ed6-bce3-4cdf25a3d086 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.607994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0c577-56d3-4ed6-bce3-4cdf25a3d086 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.608492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0c577-56d3-4ed6-bce3-4cdf25a3d086 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.610260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 481/972] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:34.611229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0c577-56d3-4ed6-bce3-4cdf25a3d086 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:34.649187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:34.649963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.650248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.650523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.651243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:34.652704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:34.667360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.sql.core [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:11:34.667360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:11:34.667360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [parameters: {'user_id': 'c32db74f0b5144c9b357d72c1a9069c5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:11:34.667360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80273) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-9b7f0283-eea1-4aaf-902b-8b329338a012 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application conn.query(q) May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result.read() May 27 03:11:34.678405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:11:34.679355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application next(self.gen) May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application yield resource May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application next(self.gen) May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise self.value May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application session.commit() May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:11:34.680412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.session.flush() May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application compat.raise_( May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:11:34.681322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:11:34.682273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application conn.query(q) May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application result.read() May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:11:34.683130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c32db74f0b5144c9b357d72c1a9069c5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:34.685587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 03:11:34.687734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 492/973] 10.222.0.62 () {60 vars in 1131 bytes} [Fri May 27 03:11:34 2022] POST /identity/v3/users => generated 160 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b00af-d638-4ae2-b808-2769eafcf8e2 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b00af-d638-4ae2-b808-2769eafcf8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.690780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b00af-d638-4ae2-b808-2769eafcf8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.708007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b19b00af-d638-4ae2-b808-2769eafcf8e2 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:34.709476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b19b00af-d638-4ae2-b808-2769eafcf8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:34.730226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0c577-56d3-4ed6-bce3-4cdf25a3d086 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:34.755618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:34.774909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:34.775726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.776031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.776303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.785935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 493/974] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:34.786956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b00af-d638-4ae2-b808-2769eafcf8e2 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:34.794610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:34.796914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:34.824375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:34.825360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:34.826276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:34.826276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:34.826276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:34.826276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:34.826917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 482/975] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:34.827430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:34.851898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:34.891253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b00af-d638-4ae2-b808-2769eafcf8e2 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:34.928902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:34.930012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.930733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.931005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.939196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:34.940914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:34.956170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:34.957337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:11:34.958535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:34.959222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd133ff-af02-4348-bfcb-1ad47b1d315a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:34.962320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 494/976] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:34.963119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:34.993521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c51371-0da8-46f9-aae9-c82d70e2cebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:34.994324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c51371-0da8-46f9-aae9-c82d70e2cebc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:34.994608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c51371-0da8-46f9-aae9-c82d70e2cebc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:34.994883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c51371-0da8-46f9-aae9-c82d70e2cebc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:34.995538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c51371-0da8-46f9-aae9-c82d70e2cebc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:34.996951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c51371-0da8-46f9-aae9-c82d70e2cebc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.002290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-92c51371-0da8-46f9-aae9-c82d70e2cebc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:35.009054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:35.010285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:35.011126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 483/977] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:35.011983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05354e-92ce-47e4-8772-e70420de794b None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.011983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05354e-92ce-47e4-8772-e70420de794b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.012117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05354e-92ce-47e4-8772-e70420de794b None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.013308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 484/978] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:35.013998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05354e-92ce-47e4-8772-e70420de794b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.066988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2355c-f5b0-4432-9ce8-4c59e50cd4bb None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.090689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.101393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.102063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.102339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.102610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.103336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:35.103642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.105036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.118783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 495/979] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.119557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.119659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05354e-92ce-47e4-8772-e70420de794b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.155624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fdc96-b157-4d77-81af-d944e91828a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.156316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fdc96-b157-4d77-81af-d944e91828a7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.156596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fdc96-b157-4d77-81af-d944e91828a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.156869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fdc96-b157-4d77-81af-d944e91828a7 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.157667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fdc96-b157-4d77-81af-d944e91828a7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:35.157971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fdc96-b157-4d77-81af-d944e91828a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.160047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fdc96-b157-4d77-81af-d944e91828a7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.192900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 485/980] 10.222.0.62 () {56 vars in 1101 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.194120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f604-a145-45d8-a2d1-f724285c0210 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.194407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f604-a145-45d8-a2d1-f724285c0210 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.194682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f604-a145-45d8-a2d1-f724285c0210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.217936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.220019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d254f604-a145-45d8-a2d1-f724285c0210 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:35.221352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d254f604-a145-45d8-a2d1-f724285c0210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:35.223467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b464bb-25cf-4cf3-bb4e-3050267c473b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.263271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.264006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.264292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.264566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.265289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:35.265598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.266992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.273298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 496/981] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.274033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.303864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 486/982] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:35.303864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254f604-a145-45d8-a2d1-f724285c0210 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.377369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.394386 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbc5c6-087c-464f-9a7f-eb0aa9ec12dd None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.424742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254f604-a145-45d8-a2d1-f724285c0210 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.437228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.438048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.438331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.438611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.446272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.447707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:35.462726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:35.463909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 497/983] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:35.465069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.469272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.469878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.470159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.470434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.471150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:35.471490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.472893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.478198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 487/984] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.479949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.490986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.564792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c07211-c8f1-49a3-90c1-7a00f3479a2d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.577359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5314d-fe5b-4430-af23-fb1c1efa488b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.599493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.600179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.600466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.600747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.601472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.602841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.603626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.612910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.613566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.613853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.614130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.614923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:35.615233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.616339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 498/985] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:35 2022] POST /identity/v3/services => generated 242 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:35.616710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.617144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.634349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 488/986] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.635749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.727598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.746587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526b0d-72bb-43a1-b90c-26bd1c981a48 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.757837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82313b-d138-4eb9-a2e7-80d22091feb3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.783913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.784584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.784867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.785143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.785993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] RBAC: Adding query filter params (role.id=f88a7c60dc0b4604be9ca139688c9ae7, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=1c908d4a33a041d8acc6ecad8c39ac6f) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:35.786304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.787786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.795349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 489/987] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/role_assignments?user.id=1c908d4a33a041d8acc6ecad8c39ac6f&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=f88a7c60dc0b4604be9ca139688c9ae7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.796348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732b1f-6d30-4641-843a-613ca92f3814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.797019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732b1f-6d30-4641-843a-613ca92f3814 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.797307 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732b1f-6d30-4641-843a-613ca92f3814 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.797589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732b1f-6d30-4641-843a-613ca92f3814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.798416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732b1f-6d30-4641-843a-613ca92f3814 None admin] RBAC: Adding query filter params (interface=admin, service_id=554df699ac514b5aaf6b8f04c38fb304, region_id=RegionOne) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:35.798739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732b1f-6d30-4641-843a-613ca92f3814 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.800724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732b1f-6d30-4641-843a-613ca92f3814 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.807704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 499/988] 10.222.0.62 () {56 vars in 1249 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/endpoints?service_id=554df699ac514b5aaf6b8f04c38fb304&interface=admin®ion_id=RegionOne => generated 193 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.816106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.877359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.941937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7ff6-ae73-426d-bcae-9ce7e7825355 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.979624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef9974-de9e-4e5c-8b8e-91817360d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.980418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef9974-de9e-4e5c-8b8e-91817360d6eb None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.980814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef9974-de9e-4e5c-8b8e-91817360d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.981192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef9974-de9e-4e5c-8b8e-91817360d6eb None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.982003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef9974-de9e-4e5c-8b8e-91817360d6eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.983482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef9974-de9e-4e5c-8b8e-91817360d6eb None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.992136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 490/989] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:11:37.876330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd1434-2a89-4c1c-9f8d-5662a7a9ff6a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:37.876886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd1434-2a89-4c1c-9f8d-5662a7a9ff6a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:37.876886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd1434-2a89-4c1c-9f8d-5662a7a9ff6a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:37.878944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 500/990] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:37.884044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc8e32-e5fb-4ebb-ad3a-67c11c07615f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:37.884448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc8e32-e5fb-4ebb-ad3a-67c11c07615f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:37.884627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc8e32-e5fb-4ebb-ad3a-67c11c07615f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:37.901674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-21fc8e32-e5fb-4ebb-ad3a-67c11c07615f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:37.902846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-21fc8e32-e5fb-4ebb-ad3a-67c11c07615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:37.979877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 491/991] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:38.003111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd1434-2a89-4c1c-9f8d-5662a7a9ff6a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.109440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd1434-2a89-4c1c-9f8d-5662a7a9ff6a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.142748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef9ebe-f524-4d37-80d8-d8efb129c157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.143500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef9ebe-f524-4d37-80d8-d8efb129c157 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.143779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef9ebe-f524-4d37-80d8-d8efb129c157 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.144082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef9ebe-f524-4d37-80d8-d8efb129c157 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.144738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef9ebe-f524-4d37-80d8-d8efb129c157 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.146101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef9ebe-f524-4d37-80d8-d8efb129c157 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-28ef9ebe-f524-4d37-80d8-d8efb129c157 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:38.157756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:38.158747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:38.159589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 501/992] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:38.166428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc8e32-e5fb-4ebb-ad3a-67c11c07615f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.179889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524d87-dcd6-4158-967f-0374e1c35bbf None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.180187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524d87-dcd6-4158-967f-0374e1c35bbf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.180474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524d87-dcd6-4158-967f-0374e1c35bbf None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.181969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 502/993] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:38.186209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471445e2-eb7b-46ef-a5a6-97ae86206625 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.186501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471445e2-eb7b-46ef-a5a6-97ae86206625 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.186783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471445e2-eb7b-46ef-a5a6-97ae86206625 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.188070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 503/994] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:38.189196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae2b96-b511-4c4c-8381-2b0bff147d10 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.189384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae2b96-b511-4c4c-8381-2b0bff147d10 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.189664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae2b96-b511-4c4c-8381-2b0bff147d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.201591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.207423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-5aae2b96-b511-4c4c-8381-2b0bff147d10 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:38.208632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5aae2b96-b511-4c4c-8381-2b0bff147d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:38.213998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.267033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc8e32-e5fb-4ebb-ad3a-67c11c07615f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.274417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 504/995] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:38.275495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3f1a-5a52-4844-9e54-c558e2ecf169 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.275785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3f1a-5a52-4844-9e54-c558e2ecf169 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.276092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3f1a-5a52-4844-9e54-c558e2ecf169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.291833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-dacc3f1a-5a52-4844-9e54-c558e2ecf169 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:38.292998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-dacc3f1a-5a52-4844-9e54-c558e2ecf169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:38.305486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.306150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.306438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.306715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.307432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:38.307743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.309143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.313856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 492/996] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.314618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.359625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 505/997] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:38.360469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3f1a-5a52-4844-9e54-c558e2ecf169 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.415637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40640417-e01d-4ddd-bf34-2df293f3a417 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.426699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.431597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.451538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.452357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.452644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.452924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] PATH_INFO: `/v3/services/image` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.453580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.454950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.465390 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3f1a-5a52-4844-9e54-c558e2ecf169 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:38.466497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:38.467477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:38.468984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 493/998] 10.222.0.62 () {56 vars in 1105 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:38.469817 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.503190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.503881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.504171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.504448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.505158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:38.505470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.506832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.511709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 506/999] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.512415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.570090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a7a67-f13f-442a-862b-10c7b85f860d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.602948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.603747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.604061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.604345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.604999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.606366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.608013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37483d01-b09e-4f12-8f86-8cb7fe5f1a3c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:38.619626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:38.620707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:11:38.621539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:38.623958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 494/1000] 10.222.0.62 () {56 vars in 1126 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:38.624963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.637376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.643667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.644380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.644664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.644939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.645649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] RBAC: Adding query filter params (name=image) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:38.645959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.647321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.652069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 507/1001] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.652828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.668933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.729133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c870-296c-4e91-9b28-2bb7f29af221 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.753776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca773f-5e33-4976-8cdc-b4d5fa734986 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.761742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.762398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.762682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.762959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.763751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] RBAC: Adding query filter params (interface=public, service_id=2a08cc1cb8284f2d94adb30665a7c202, region_id=RegionOne) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:38.764090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.765459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.770944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 495/1002] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/endpoints?service_id=2a08cc1cb8284f2d94adb30665a7c202&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.771723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.789134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.789796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.790081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.790361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.791079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:38.791395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.792826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.797577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 508/1003] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.798343 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.854898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.870569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f19da-cf88-4a7b-bc96-dd9f66a5d8a9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.872103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.895871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc0739-1087-4103-85d0-6585d7391b0f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.904613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.905265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.905550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.905828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.906536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] RBAC: Adding query filter params (type=image) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:38.906845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.908237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.913352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 496/1004] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.913959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.939517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.940325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.940613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.941276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.942193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.943943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.963851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 509/1005] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:11:39.044763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110656a8-7be8-491b-997c-30ff8be5be62 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:39.083389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163a10e-3904-453e-93b5-eee74b237108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:39.084110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a10e-3904-453e-93b5-eee74b237108 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:39.084397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a10e-3904-453e-93b5-eee74b237108 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:39.084672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a10e-3904-453e-93b5-eee74b237108 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:39.085907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163a10e-3904-453e-93b5-eee74b237108 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:39.086219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163a10e-3904-453e-93b5-eee74b237108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:39.087995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163a10e-3904-453e-93b5-eee74b237108 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:39.096758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 497/1006] 10.222.0.62 () {56 vars in 1122 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:39.100449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:39.211268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802c0f9-ecc4-47da-8aee-3d9266f6935d None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:39.234127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:39.246231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d8bc7-d6ad-47a8-9e5c-ad3c35320879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:39.246896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8bc7-d6ad-47a8-9e5c-ad3c35320879 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:39.247182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8bc7-d6ad-47a8-9e5c-ad3c35320879 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:39.247460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8bc7-d6ad-47a8-9e5c-ad3c35320879 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:39.248252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d8bc7-d6ad-47a8-9e5c-ad3c35320879 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:39.249626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d8bc7-d6ad-47a8-9e5c-ad3c35320879 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:39.266317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 510/1007] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:11:41.141744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b80267-010a-4e7f-8a11-7031283808b7 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.142290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b80267-010a-4e7f-8a11-7031283808b7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.142290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b80267-010a-4e7f-8a11-7031283808b7 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.143880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 498/1008] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:41.149008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a243c-dba4-4333-bb91-ddba4ed70117 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.149386 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a243c-dba4-4333-bb91-ddba4ed70117 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.149569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a243c-dba4-4333-bb91-ddba4ed70117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.166446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-072a243c-dba4-4333-bb91-ddba4ed70117 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:41.167881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-072a243c-dba4-4333-bb91-ddba4ed70117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:41.168013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e8fd6-369d-42c2-ad1e-27df97ab31c8 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.168210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e8fd6-369d-42c2-ad1e-27df97ab31c8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.168493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e8fd6-369d-42c2-ad1e-27df97ab31c8 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.169874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 499/1009] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:41.175446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bbd41-5ba2-4ce7-b032-e74cd53439a3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.175740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bbd41-5ba2-4ce7-b032-e74cd53439a3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.176054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bbd41-5ba2-4ce7-b032-e74cd53439a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.193030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-498bbd41-5ba2-4ce7-b032-e74cd53439a3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:41.194126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-498bbd41-5ba2-4ce7-b032-e74cd53439a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:41.217745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.246224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 511/1010] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:41.266068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 500/1011] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:41.273209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a243c-dba4-4333-bb91-ddba4ed70117 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.288621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bbd41-5ba2-4ce7-b032-e74cd53439a3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.386963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a243c-dba4-4333-bb91-ddba4ed70117 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.394852 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bbd41-5ba2-4ce7-b032-e74cd53439a3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.396502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.426656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.427326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.427615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.427928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.428670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.430046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.432840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc79f-c54d-4362-a4c1-b2b13e02d159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.433638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc79f-c54d-4362-a4c1-b2b13e02d159 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.433929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc79f-c54d-4362-a4c1-b2b13e02d159 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.434295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc79f-c54d-4362-a4c1-b2b13e02d159 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.435068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc79f-c54d-4362-a4c1-b2b13e02d159 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.436620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc79f-c54d-4362-a4c1-b2b13e02d159 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-cfbcc79f-c54d-4362-a4c1-b2b13e02d159 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:41.449095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:41.450181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:11:41.451055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:41.451812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 512/1012] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:41 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:11:41.455608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 501/1013] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:41.456697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.467001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa13902-fad2-4ef2-90b4-c0605c659686 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.467001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa13902-fad2-4ef2-90b4-c0605c659686 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.467283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa13902-fad2-4ef2-90b4-c0605c659686 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.468881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 502/1014] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:41.475108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635c0e7-4340-4267-92bb-d30ef367683a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.475413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635c0e7-4340-4267-92bb-d30ef367683a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.475749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635c0e7-4340-4267-92bb-d30ef367683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.494404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-5635c0e7-4340-4267-92bb-d30ef367683a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:41.495609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5635c0e7-4340-4267-92bb-d30ef367683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:41.517177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.581070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5277bb8-c96a-4563-8b66-95f4d4ac06e6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.585020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 503/1015] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:41 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:41.609392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635c0e7-4340-4267-92bb-d30ef367683a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.620736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.621407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.621691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.621970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.622989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:41.623098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.624470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.630836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 513/1016] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:41.636475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.668122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.730054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635c0e7-4340-4267-92bb-d30ef367683a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.749747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a9e95-85d5-4a15-ac55-506d480cd5f3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.767345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.768732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.769018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.769297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.770502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.771997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.775866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 504/1017] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:41.779491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.786703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.787373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.787661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.787974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.788693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:41.789001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.790366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.795373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 514/1018] 10.222.0.62 () {56 vars in 1112 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:41.799412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.801203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.886822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d59da-be03-407f-9a64-f0fda8bff4aa None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.901207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.911290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214dfab-a198-44b3-a80c-0143eb996b7f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.933380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b86eb-53e4-4392-8558-dd6bb0cf2fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.933380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b86eb-53e4-4392-8558-dd6bb0cf2fd8 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.933380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b86eb-53e4-4392-8558-dd6bb0cf2fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.933380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b86eb-53e4-4392-8558-dd6bb0cf2fd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.933836 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b86eb-53e4-4392-8558-dd6bb0cf2fd8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.933935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b86eb-53e4-4392-8558-dd6bb0cf2fd8 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.951576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47e9ab-1000-4d6a-bceb-af55b1eb78a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.952682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47e9ab-1000-4d6a-bceb-af55b1eb78a9 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.952982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47e9ab-1000-4d6a-bceb-af55b1eb78a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.953308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47e9ab-1000-4d6a-bceb-af55b1eb78a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.954263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47e9ab-1000-4d6a-bceb-af55b1eb78a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.955775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47e9ab-1000-4d6a-bceb-af55b1eb78a9 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.964471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 505/1019] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:11:41 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:11:41.981334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 515/1020] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:11:43.650097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fb91d-15d4-4032-ac69-fae21642a3f5 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:43.650624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fb91d-15d4-4032-ac69-fae21642a3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:43.650705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fb91d-15d4-4032-ac69-fae21642a3f5 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:43.652074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 506/1021] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:43.657162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db49f-a477-4bf5-b1d5-93b009bd548b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:43.657573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db49f-a477-4bf5-b1d5-93b009bd548b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:43.657737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9db49f-a477-4bf5-b1d5-93b009bd548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:43.675286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9c9db49f-a477-4bf5-b1d5-93b009bd548b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:43.676475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9c9db49f-a477-4bf5-b1d5-93b009bd548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:43.750868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 516/1022] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:43.772743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fb91d-15d4-4032-ac69-fae21642a3f5 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:43.845088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:43.880321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fb91d-15d4-4032-ac69-fae21642a3f5 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:43.913657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:43.914362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:43.914642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:43.914916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] PATH_INFO: `/v3/services/image` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:43.915573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:43.918822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:43.933320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:43.934343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:43.935226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:43.938032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 507/1023] 10.222.0.62 () {56 vars in 1105 bytes} [Fri May 27 03:11:43 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:43.939283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9db49f-a477-4bf5-b1d5-93b009bd548b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.016902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.047322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9db49f-a477-4bf5-b1d5-93b009bd548b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.084820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86918e2-73c0-40dc-b00b-591f78948982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.085492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86918e2-73c0-40dc-b00b-591f78948982 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.085780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86918e2-73c0-40dc-b00b-591f78948982 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.086061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86918e2-73c0-40dc-b00b-591f78948982 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.087047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86918e2-73c0-40dc-b00b-591f78948982 None admin] RBAC: Adding query filter params (name=image) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:44.087359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86918e2-73c0-40dc-b00b-591f78948982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.088780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86918e2-73c0-40dc-b00b-591f78948982 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.098679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 517/1024] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:44.099901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.205701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50b4f-5be4-4d48-bf39-e511b9142c2f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.238980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800247f-0aad-401e-a35a-a71447714456 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.239313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800247f-0aad-401e-a35a-a71447714456 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.239549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800247f-0aad-401e-a35a-a71447714456 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.239956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.240624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.241098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.241214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 518/1025] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:44.241293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.242020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] RBAC: Adding query filter params (type=image) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:44.242331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.243703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.245138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.245834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc7ce4-95a1-40d1-b06f-a0afb2fb5eef None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.246219 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc7ce4-95a1-40d1-b06f-a0afb2fb5eef None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.246393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc7ce4-95a1-40d1-b06f-a0afb2fb5eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.251031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 508/1026] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:44.256302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.266707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-1edc7ce4-95a1-40d1-b06f-a0afb2fb5eef None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:44.268451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1edc7ce4-95a1-40d1-b06f-a0afb2fb5eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:44.341470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 519/1027] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:44.342777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211623d0-190d-4cda-a37d-9f093679940b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.343064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211623d0-190d-4cda-a37d-9f093679940b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.343342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211623d0-190d-4cda-a37d-9f093679940b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.347861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 520/1028] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:44.350259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bf23b-828f-4b2f-aaaa-47f64dabbfbf None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.350690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bf23b-828f-4b2f-aaaa-47f64dabbfbf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.351120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bf23b-828f-4b2f-aaaa-47f64dabbfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.357083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d59a0-e89f-4291-a9f6-9d0a36777aaf None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.367995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-934bf23b-828f-4b2f-aaaa-47f64dabbfbf None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:44.369409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-934bf23b-828f-4b2f-aaaa-47f64dabbfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:44.390397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.391053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.391342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.391621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.392966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] RBAC: Adding query filter params (interface=public, service_id=0091f4e8e822411fa1cb8004793afc83, region_id=RegionOne) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:44.393280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.394642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.399497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 509/1029] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/endpoints?service_id=0091f4e8e822411fa1cb8004793afc83&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:44.400304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.431248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.437414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 521/1030] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:44.438223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bf23b-828f-4b2f-aaaa-47f64dabbfbf None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.464532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.502649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fbdf6-a706-4d41-9c16-4358ec6c9670 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.535837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457978db-a3bc-487b-9934-149433f615cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.536614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457978db-a3bc-487b-9934-149433f615cb None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.536901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457978db-a3bc-487b-9934-149433f615cb None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.537174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457978db-a3bc-487b-9934-149433f615cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.538340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457978db-a3bc-487b-9934-149433f615cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.539765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457978db-a3bc-487b-9934-149433f615cb None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.541399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 510/1031] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:44.542154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457978db-a3bc-487b-9934-149433f615cb None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.542474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bf23b-828f-4b2f-aaaa-47f64dabbfbf None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.579251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.580481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.580773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.581447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.582140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.583869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.590562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 522/1032] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:11:44.591312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.649217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457978db-a3bc-487b-9934-149433f615cb None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.660036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.681405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.688498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.689861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.690166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.690422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.702532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:44.707860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8b2ae-036f-4d40-855b-d8cc230013b8 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:44.712138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:44.713153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:44.714030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:44.719018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 511/1033] 10.222.0.62 () {56 vars in 1120 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:44.719761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.754863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.756184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.756701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.756902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.757996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.759471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.781988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 523/1034] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:11:44 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:44.838623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1298db2-ae23-41b2-873b-4d84c0a9ba4c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.878742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.879437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.879723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.880087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.880876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:44.881189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.882679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.892549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 512/1035] 10.222.0.62 () {56 vars in 1115 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:44.898342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.015560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40eb77-0634-4b87-992e-a73e073dd707 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.034227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.054021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.054819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.055104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.055380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.066344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.067797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:45.085478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:45.086691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:45.087608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:45.087608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:45.087608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:45.087608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:45.087608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 524/1036] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:45.092946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.148496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.198071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9b485-1c52-4ee0-9d08-fa9c82a11cab None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.231919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.232580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.232866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.233200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.233929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:45.234236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.235610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.247611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 513/1037] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:45.253491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.354628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b71569-f18c-42d9-82b5-8311b309a5b6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.388626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.389417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.389701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.389979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.398114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.399554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:45.413530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:45.414506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:45.415470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:45.419850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 525/1038] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:45.420549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.522778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84ad2d-2355-4209-8278-6398c28188de None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.535891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.557726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.558445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.558729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.559059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.559929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:45.560246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.561667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.583780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 514/1039] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:45.585299 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.617574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.689950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721369e-f6eb-4e0f-88c3-83d607e46739 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.724917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.725581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.725867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.726146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.793225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.794668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.823910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.898643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 526/1040] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:11:45.899948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:46.013310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586d707-3791-4692-96cb-67a78065d996 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:46.047200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:46.048025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:46.048308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:46.049201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:46.059251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:46.060755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:46.078376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:46.079320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:46.080791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:46.080791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:46.080791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:46.080791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:46.084129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 515/1041] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:46.085345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:46.187352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2df564-25e7-4a32-bd1c-1b1c44ddac3e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:46.215707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:46.222853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:46.223519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:46.223802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:46.224106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:46.224835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:46.225145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:46.226505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:46.242702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 527/1042] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:46.243893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:46.268757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:46.344474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559c6e-5617-4ef8-876f-63da086b46a2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:46.378729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:46.379521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:46.379799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:46.380100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:46.386800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:46.388281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:46.402298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:46.403210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:46.409625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 516/1043] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:46.411757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:46.512894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c0870-cad2-49b9-9e92-4a42435be6d1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:46.547056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395412-9595-4358-9306-243bf19165d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:46.547726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395412-9595-4358-9306-243bf19165d3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:46.548040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395412-9595-4358-9306-243bf19165d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:46.548318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395412-9595-4358-9306-243bf19165d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:46.549754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33395412-9595-4358-9306-243bf19165d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:46.550070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33395412-9595-4358-9306-243bf19165d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:46.551496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33395412-9595-4358-9306-243bf19165d3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:46.569006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 528/1044] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:46.570380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:46.658701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:46.679475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d6ab-ac2b-4f55-b3b7-2fa32ba2c3d9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:46.714259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d986eae-6085-4186-aa95-bfadba3595c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:46.714977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986eae-6085-4186-aa95-bfadba3595c6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:46.715265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986eae-6085-4186-aa95-bfadba3595c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:46.715587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d986eae-6085-4186-aa95-bfadba3595c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:46.716596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d986eae-6085-4186-aa95-bfadba3595c6 None admin] RBAC: Adding query filter params (role.id=f88a7c60dc0b4604be9ca139688c9ae7, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=edc938bed535461e974daf01bb99e9a6) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:46.716977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d986eae-6085-4186-aa95-bfadba3595c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:46.718491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d986eae-6085-4186-aa95-bfadba3595c6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:46.727913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 517/1045] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:46 2022] GET /identity/v3/role_assignments?user.id=edc938bed535461e974daf01bb99e9a6&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=f88a7c60dc0b4604be9ca139688c9ae7 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:46.808569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e3847-8049-4efc-82c4-34a6f1588e8c None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:46.809114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e3847-8049-4efc-82c4-34a6f1588e8c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:46.809114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e3847-8049-4efc-82c4-34a6f1588e8c None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:46.810574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 529/1046] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:46.816133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd69b0-5b31-4eb9-9da2-c0d5e028c334 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:46.816426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd69b0-5b31-4eb9-9da2-c0d5e028c334 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:46.816698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd69b0-5b31-4eb9-9da2-c0d5e028c334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:46.834081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b5fd69b0-5b31-4eb9-9da2-c0d5e028c334 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:46.835297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b5fd69b0-5b31-4eb9-9da2-c0d5e028c334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:46.913115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 518/1047] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:46.936000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e3847-8049-4efc-82c4-34a6f1588e8c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:46.989453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:47.041686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e3847-8049-4efc-82c4-34a6f1588e8c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.069776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44042a5-7c5b-48bf-a1b7-9e5151a88dbe None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.069999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44042a5-7c5b-48bf-a1b7-9e5151a88dbe None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.070285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44042a5-7c5b-48bf-a1b7-9e5151a88dbe None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.071662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 519/1048] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:47.076014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.076748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e118a5-ab7c-4d36-b9f5-39bc0f29fa4a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.076983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.077088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e118a5-ab7c-4d36-b9f5-39bc0f29fa4a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.077262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.077360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e118a5-ab7c-4d36-b9f5-39bc0f29fa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.077540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] PATH_INFO: `/v3/services/image` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.078199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:47.080096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:47.093451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:47.094674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:47.096182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 530/1049] 10.222.0.62 () {56 vars in 1105 bytes} [Fri May 27 03:11:46 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:47.100907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-a2e118a5-ab7c-4d36-b9f5-39bc0f29fa4a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:47.101184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:47.102100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a2e118a5-ab7c-4d36-b9f5-39bc0f29fa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:47.175561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 520/1050] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:47.198732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e118a5-ab7c-4d36-b9f5-39bc0f29fa4a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:47.204558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60f5f8-f4ba-4a13-8b11-f70c49693519 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.207175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:47.222802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:47.243922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.244591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.244863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.245318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.245843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:47.246150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:47.247562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:47.252734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 531/1051] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:47.258277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:47.307738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e118a5-ab7c-4d36-b9f5-39bc0f29fa4a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.344322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.345113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.345940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.346214 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.356952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:47.364536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:47.365483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:47.368718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 521/1052] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:47.369419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b418e-9c7e-4412-963b-d9811d8a0ba3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.371798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:47.402478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.403153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.403443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.403728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.404476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] RBAC: Adding query filter params (type=image) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:47.404789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:47.406158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:47.416489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 532/1053] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:47.417506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:47.443007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:47.465780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:47.479460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa48583-f9e9-40a5-9077-0cef3bf3a46f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.515598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.516295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.516577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.516853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.518211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:47.518515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:47.519923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:47.528004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50018d-e8d9-40cb-a504-c76b0ac8fe73 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.528669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 522/1054] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:47.534072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:47.563772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.565080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.565371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.565696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.566460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:47.567895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:47.587674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 533/1055] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:11:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:47.643912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73a44-d766-4c66-a31e-f8c3b300cbb6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.674130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:47.690095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.690996 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.691281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.691559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] PATH_INFO: `/v3/users/5c25beb62d9f49e9accd9462f3ad2ba4` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.707339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] RBAC: Authorizing `identity:get_user(user_id=5c25beb62d9f49e9accd9462f3ad2ba4)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:47.708858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:47.710985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 523/1056] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:11:47 2022] GET /identity/v3/users/5c25beb62d9f49e9accd9462f3ad2ba4 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:47.716853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:47.824889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:47.830029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1351d-c5cc-4cbf-8807-bd04ae690433 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:47.867792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:47.868479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:47.868760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:47.869035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:47.944498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:47.945975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:48.042810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:48.053268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 534/1057] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:48.054243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:48.154807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1788f-ccf7-4184-9278-7573f392f29a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:48.188487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217fdd1-b8b3-473f-b5a3-d81fcd0c1716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:48.189302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217fdd1-b8b3-473f-b5a3-d81fcd0c1716 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:48.189582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217fdd1-b8b3-473f-b5a3-d81fcd0c1716 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:48.189855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217fdd1-b8b3-473f-b5a3-d81fcd0c1716 None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:48.191021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217fdd1-b8b3-473f-b5a3-d81fcd0c1716 None admin] RBAC: Authorizing `identity:get_project(project_id=ba58af6fbb574b2b85b5129022ebd56f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:48.192478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217fdd1-b8b3-473f-b5a3-d81fcd0c1716 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:48.194312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 524/1058] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:11:48 2022] GET /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:48.200059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:48.297087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f4256-1564-4d11-b0cf-229c36bf5e89 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:48.331501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55783b46-2dc2-4566-8acc-b89b84ec8f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:48.332190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55783b46-2dc2-4566-8acc-b89b84ec8f7e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:48.332474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55783b46-2dc2-4566-8acc-b89b84ec8f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:48.332752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55783b46-2dc2-4566-8acc-b89b84ec8f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:48.333594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55783b46-2dc2-4566-8acc-b89b84ec8f7e None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=ba58af6fbb574b2b85b5129022ebd56f, user.id=5c25beb62d9f49e9accd9462f3ad2ba4) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:48.333907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55783b46-2dc2-4566-8acc-b89b84ec8f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:48.335326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55783b46-2dc2-4566-8acc-b89b84ec8f7e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:48.347860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 535/1059] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:48 2022] GET /identity/v3/role_assignments?user.id=5c25beb62d9f49e9accd9462f3ad2ba4&scope.project.id=ba58af6fbb574b2b85b5129022ebd56f&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:48.939847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e842b4-5e13-4aa4-8bf7-e8a099e5694d None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:48.940159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e842b4-5e13-4aa4-8bf7-e8a099e5694d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:48.940438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e842b4-5e13-4aa4-8bf7-e8a099e5694d None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:48.941896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 525/1060] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:48.946975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60022f3d-257c-4b8f-a49f-f6f284492699 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:48.947403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60022f3d-257c-4b8f-a49f-f6f284492699 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:48.947580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60022f3d-257c-4b8f-a49f-f6f284492699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:48.964249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-60022f3d-257c-4b8f-a49f-f6f284492699 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:48.965411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-60022f3d-257c-4b8f-a49f-f6f284492699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:49.021436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:49.033786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 536/1061] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:49.056124 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e842b4-5e13-4aa4-8bf7-e8a099e5694d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:49.106704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:49.159935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e842b4-5e13-4aa4-8bf7-e8a099e5694d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:49.194348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:49.195111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:49.195391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:49.195664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:49.208134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:49.216588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:49.217533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:49.222837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 526/1062] 10.222.0.62 () {56 vars in 1120 bytes} [Fri May 27 03:11:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:49.226234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60022f3d-257c-4b8f-a49f-f6f284492699 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:49.325692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60022f3d-257c-4b8f-a49f-f6f284492699 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:49.358908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:49.359580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:49.359887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:49.360167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:49.361440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:49.361748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:49.363102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:49.376074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 537/1063] 10.222.0.62 () {56 vars in 1115 bytes} [Fri May 27 03:11:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:49.377960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:49.481509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb1fd-9a87-4408-9f9f-104955b9d861 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:49.492366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:49.517737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:49.518769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:49.519215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:49.519633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:49.531471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:49.533122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:49.551274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:49.552419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:49.553319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:49.553319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:49.553319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:49.553319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:49.555966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 527/1064] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:49.558815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:49.616746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:49.666774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8c72-c0a5-4387-be92-435431f3704a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:49.701006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6bcb7-86c0-4ca4-ae17-c310b4cf53bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:49.701670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bcb7-86c0-4ca4-ae17-c310b4cf53bc None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:49.701953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bcb7-86c0-4ca4-ae17-c310b4cf53bc None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:49.702228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bcb7-86c0-4ca4-ae17-c310b4cf53bc None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:49.702951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6bcb7-86c0-4ca4-ae17-c310b4cf53bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:49.703260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6bcb7-86c0-4ca4-ae17-c310b4cf53bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:49.705224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6bcb7-86c0-4ca4-ae17-c310b4cf53bc None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:49.718090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 538/1065] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:49.723590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:49.823402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7c217-6c47-4dfa-a80a-b11d2a2d191a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:49.828685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fb12d-6675-4921-a1fb-cd195c8b425a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:49.829204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fb12d-6675-4921-a1fb-cd195c8b425a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:49.830288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fb12d-6675-4921-a1fb-cd195c8b425a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:49.832738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 539/1066] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:49.838511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047c0f-5d9d-4baf-bdd0-212cc64f4120 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:49.839051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047c0f-5d9d-4baf-bdd0-212cc64f4120 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:49.839755 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047c0f-5d9d-4baf-bdd0-212cc64f4120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:49.859800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:49.860638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:49.860853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-36047c0f-5d9d-4baf-bdd0-212cc64f4120 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:49.860980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:49.861181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:49.862530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-36047c0f-5d9d-4baf-bdd0-212cc64f4120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:49.868923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:49.870398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:49.885148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:49.886098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:49.887088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 528/1067] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:11:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:49.892730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:49.923509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:49.943706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 540/1068] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:49.967180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36047c0f-5d9d-4baf-bdd0-212cc64f4120 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.010443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7baf35-1a06-4ae3-891e-3e893245ee6c None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.019175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.061780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.062566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.063107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.063399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.064885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:50.065524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.066977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.085469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 529/1069] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:11:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:50.088113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36047c0f-5d9d-4baf-bdd0-212cc64f4120 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.091231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.126001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.127660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.128198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.128649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.176723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.198450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcc1a9-4a32-473a-92ba-35461300022b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.202405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.204092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.234405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.235250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.235477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.235786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.245916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.247366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.253457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:50.265213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:50.266224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:50.267773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:50.267773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:50.267773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:50.267773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:50.267773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 530/1070] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:50.272690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.315970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 541/1071] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:11:50.321580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.382651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ebbc0-66ff-4530-a346-d6daa001b824 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.414940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.421327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.422260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.422560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.422841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.424303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:50.424946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.426744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.433031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf0f56-a9bf-40f6-b4d9-85edd4358797 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.440361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 531/1072] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:50.445498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.472765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.473557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.473845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.474124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.484548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.486537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.489472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:50.507462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:50.508466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:50.509394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:50.509394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:50.509394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:50.509394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:50.513837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 542/1073] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:50.514552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.559540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45730e-c574-4111-b84a-99a5f91ec3fd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.592517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954b1ca-e2b9-4dad-bfcb-0042f08ff32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.593398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954b1ca-e2b9-4dad-bfcb-0042f08ff32a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.593682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954b1ca-e2b9-4dad-bfcb-0042f08ff32a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.593960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954b1ca-e2b9-4dad-bfcb-0042f08ff32a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.601022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954b1ca-e2b9-4dad-bfcb-0042f08ff32a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.602455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954b1ca-e2b9-4dad-bfcb-0042f08ff32a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.614455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9de0f5-ef7b-45a1-91f7-eaff170ebef8 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-9954b1ca-e2b9-4dad-bfcb-0042f08ff32a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:50.617049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:50.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:50.618874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 532/1074] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:50.619393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9581651-5509-4b82-af97-561d104de5f1 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.619913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9581651-5509-4b82-af97-561d104de5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.620022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9581651-5509-4b82-af97-561d104de5f1 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.621309 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 533/1075] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:50.623912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9581651-5509-4b82-af97-561d104de5f1 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.651304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03983e02-152c-4417-8e36-c7dbdbe4afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.651920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03983e02-152c-4417-8e36-c7dbdbe4afb4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.652265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03983e02-152c-4417-8e36-c7dbdbe4afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.652541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03983e02-152c-4417-8e36-c7dbdbe4afb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.653302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03983e02-152c-4417-8e36-c7dbdbe4afb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:50.653612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03983e02-152c-4417-8e36-c7dbdbe4afb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.654973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03983e02-152c-4417-8e36-c7dbdbe4afb4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.666469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 543/1076] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:50.667707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2a7e-471e-45a1-a2bd-6916bcb2c9ae None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.667997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2a7e-471e-45a1-a2bd-6916bcb2c9ae None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.668235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2a7e-471e-45a1-a2bd-6916bcb2c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.677828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.690697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3daf2a7e-471e-45a1-a2bd-6916bcb2c9ae None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:50.692069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3daf2a7e-471e-45a1-a2bd-6916bcb2c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:50.735157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9581651-5509-4b82-af97-561d104de5f1 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.752482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.779852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 544/1077] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:50.782234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.782923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.783204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.783476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.784329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:50.784647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.786102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:50.796609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2a7e-471e-45a1-a2bd-6916bcb2c9ae None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.799914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 534/1078] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:50.808307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.911733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2a7e-471e-45a1-a2bd-6916bcb2c9ae None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.931787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74379e7-27e1-4261-8027-9d924ebe9a88 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:50.950355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:50.954813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.955614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.956012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.956295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.969049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:50.976827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:50.977823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:50.978924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 545/1079] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:50.982591 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:50.983492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:50.983772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:50.984069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:50.984167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/edc938bed535461e974daf01bb99e9a6/roles/f88a7c60dc0b4604be9ca139688c9ae7` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:50.986942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=edc938bed535461e974daf01bb99e9a6, role_id=f88a7c60dc0b4604be9ca139688c9ae7)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:50.988472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:51.002518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 535/1080] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:11:50 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/edc938bed535461e974daf01bb99e9a6/roles/f88a7c60dc0b4604be9ca139688c9ae7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:11:51.035348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:51.092800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5155-3ce0-4a60-bccf-b7b318e54868 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:51.128032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:51.128714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:51.128997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:51.129276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:51.130041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:51.130349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:51.131733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:51.141001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 546/1081] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:51.147038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:51.249250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b8822-3c5e-49fb-bfe0-f990ddaa6419 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:51.281104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:51.281966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:51.282245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:51.283035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] PATH_INFO: `/v3/users/5c25beb62d9f49e9accd9462f3ad2ba4` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:51.284250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] RBAC: Authorizing `identity:get_user(user_id=5c25beb62d9f49e9accd9462f3ad2ba4)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:51.285678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:51.287291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 536/1082] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:11:51 2022] GET /identity/v3/users/5c25beb62d9f49e9accd9462f3ad2ba4 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:51.293137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:51.389923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7ed0c-40e5-49ba-b2b3-c3be175df7f0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:51.398680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:51.423747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5183b-fd84-4b34-b404-ebfe5f96c64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:51.424635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5183b-fd84-4b34-b404-ebfe5f96c64e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:51.424922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5183b-fd84-4b34-b404-ebfe5f96c64e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:51.425200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5183b-fd84-4b34-b404-ebfe5f96c64e None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:51.426393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5183b-fd84-4b34-b404-ebfe5f96c64e None admin] RBAC: Authorizing `identity:get_project(project_id=ba58af6fbb574b2b85b5129022ebd56f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:51.427845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5183b-fd84-4b34-b404-ebfe5f96c64e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:51.429567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 547/1083] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:11:51 2022] GET /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:51.435403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:51.476137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:51.534366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b63027-5b0c-47d2-9366-52ccf1b37473 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:51.565932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140d647-b868-4dee-ab91-8be30af00d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:51.566810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140d647-b868-4dee-ab91-8be30af00d07 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:51.567092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140d647-b868-4dee-ab91-8be30af00d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:51.567369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140d647-b868-4dee-ab91-8be30af00d07 None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f/users/5c25beb62d9f49e9accd9462f3ad2ba4/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:51.569326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140d647-b868-4dee-ab91-8be30af00d07 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba58af6fbb574b2b85b5129022ebd56f, user_id=5c25beb62d9f49e9accd9462f3ad2ba4, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:51.570747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140d647-b868-4dee-ab91-8be30af00d07 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:51.584372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 537/1084] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:11:51 2022] PUT /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f/users/5c25beb62d9f49e9accd9462f3ad2ba4/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:53.275530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2308e-da2c-43f7-b77e-afa19e50018f None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.276064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2308e-da2c-43f7-b77e-afa19e50018f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.276064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2308e-da2c-43f7-b77e-afa19e50018f None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:53.277673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 548/1085] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:53.283502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cebb6-325b-4953-bbe9-ccefb938dd75 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.284045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cebb6-325b-4953-bbe9-ccefb938dd75 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.284157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cebb6-325b-4953-bbe9-ccefb938dd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:53.304856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-248cebb6-325b-4953-bbe9-ccefb938dd75 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:53.306046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-248cebb6-325b-4953-bbe9-ccefb938dd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:53.378503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 538/1086] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:53.402643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2308e-da2c-43f7-b77e-afa19e50018f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:53.524177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2308e-da2c-43f7-b77e-afa19e50018f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:53.570381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be230a-fac4-4828-ac10-e469c52b5e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:53.571129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be230a-fac4-4828-ac10-e469c52b5e80 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.571421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be230a-fac4-4828-ac10-e469c52b5e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.571698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be230a-fac4-4828-ac10-e469c52b5e80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:53.583660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be230a-fac4-4828-ac10-e469c52b5e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-90be230a-fac4-4828-ac10-e469c52b5e80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:53.592128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:53.593254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:11:53.594322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:53.595981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 549/1087] 10.222.0.62 () {56 vars in 1120 bytes} [Fri May 27 03:11:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:53.599286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cebb6-325b-4953-bbe9-ccefb938dd75 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:53.628398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:53.697738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cebb6-325b-4953-bbe9-ccefb938dd75 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:53.733055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:53.733726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.734014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.734294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:53.735049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:53.735392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:53.736795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:53.737497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901d55f-c6bd-4cc6-8cc9-4806c422b187 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.737785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901d55f-c6bd-4cc6-8cc9-4806c422b187 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.738065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901d55f-c6bd-4cc6-8cc9-4806c422b187 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:53.739430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 550/1088] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:11:53.744229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5268a-5599-4a8f-b901-f81705878629 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.744516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5268a-5599-4a8f-b901-f81705878629 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.744795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5268a-5599-4a8f-b901-f81705878629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:53.744896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 539/1089] 10.222.0.62 () {56 vars in 1115 bytes} [Fri May 27 03:11:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:53.750148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:53.761417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-38d5268a-5599-4a8f-b901-f81705878629 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:53.762580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-38d5268a-5599-4a8f-b901-f81705878629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:53.791923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:53.833034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 551/1090] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:11:53.840821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:53.849794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f432-395d-4ca2-b679-b210edb02c42 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:53.856570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5268a-5599-4a8f-b901-f81705878629 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:53.884439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:53.885223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.885501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.885772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:53.896364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:53.897794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:53.914687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:53.915774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:53.916770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:53.916770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:53.916770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:53.916770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:53.917614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 540/1091] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:53.923630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:53.959547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5268a-5599-4a8f-b901-f81705878629 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:53.995134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:53.995952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:53.996243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:53.996522 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.008286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:54.015908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:54.017137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:54.018232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 552/1092] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:11:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:54.023109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.025749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96cc49-543c-462b-9b66-3f3985b631fd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.049694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.059485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.060177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.060467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.060747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.061509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:54.061822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.063200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.075346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 541/1093] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:54.080800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.084256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.123091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46edca-7ae1-4b50-94d2-8f7d7489b1c6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.156136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.156888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.157690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.157966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.158713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:54.159018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.160396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.168691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 553/1094] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:54.175085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.179452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a330-5efc-4037-93c8-c0b3b3c17bdc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.212927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.213712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.213991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.214266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.221820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.223253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:11:54.237701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:54.238668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:54.239640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 542/1095] 10.222.0.62 () {56 vars in 1093 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:11:54.244429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.260645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.273165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6d2e2-959f-4658-b490-97a9a8719d79 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.305384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.306397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.307376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.307660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.307970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] PATH_INFO: `/v3/users/5c25beb62d9f49e9accd9462f3ad2ba4` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.309187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] RBAC: Authorizing `identity:get_user(user_id=5c25beb62d9f49e9accd9462f3ad2ba4)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.310717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.312417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 554/1096] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/users/5c25beb62d9f49e9accd9462f3ad2ba4 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:54.318059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.346445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a2592-d827-40ab-9bb0-2714d3a1b0c4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.379160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.379834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.380134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.380408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.381235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:54.381540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.382972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.399477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 543/1097] 10.222.0.62 () {56 vars in 1133 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:54.405021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.413384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ccb39-a023-4246-bdc7-12b4551a6503 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.447182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.447856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.448144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.448423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.474753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.500742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0127b-be61-4ad7-9b4a-f06d41d044a6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.518009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.519456 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.524057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.537013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.537656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.537943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.538219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.604495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.605856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.614610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 555/1098] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:11:54.620393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.674853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.700514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 544/1099] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:54.706683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.718028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b112a2c-75ab-4d44-a8dc-5763ad53f200 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.729594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.751779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.752647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.752927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.753198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.754357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] RBAC: Authorizing `identity:get_project(project_id=ba58af6fbb574b2b85b5129022ebd56f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.755777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.757504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 556/1100] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:54.763327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.808280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d5f9-c55b-4ab6-b9ae-7d3124305632 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.842989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.843776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.844090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.844366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.854729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.856143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.859865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77d3b6-25f5-4465-8ece-87a18fa22535 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:54.873439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:54.874523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:54.880246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:11:54.880246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:54.880246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:11:54.880246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:11:54.880246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 545/1101] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:54.880246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:54.883129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.894535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:54.895189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:54.895476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:54.895757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:54.896631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=ba58af6fbb574b2b85b5129022ebd56f, user.id=5c25beb62d9f49e9accd9462f3ad2ba4) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:54.896945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:54.898379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:54.904191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 557/1102] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/role_assignments?user.id=5c25beb62d9f49e9accd9462f3ad2ba4&scope.project.id=ba58af6fbb574b2b85b5129022ebd56f&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:54.959280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:54.985818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f514f70-b2dd-415f-aa15-a6cee02ad12d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:55.022500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:55.023173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:55.023454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:55.023729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:55.024498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:55.024803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:55.026192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:55.038540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 546/1103] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:55.044399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:55.148030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8a36e-6e22-49b2-978b-f677daeede9b None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:55.180804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:55.181596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:55.181878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:55.182151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:55.189308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:55.190747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:11:55.210255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:55.211527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:11:55.212468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 558/1104] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:11:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:55.217734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:55.316378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb4cac-fcd9-4ff0-b109-0306e0808f40 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:55.348742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c376c90-8a14-4928-a740-a18a30c01fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:55.349395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c376c90-8a14-4928-a740-a18a30c01fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:55.349679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c376c90-8a14-4928-a740-a18a30c01fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:55.349955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c376c90-8a14-4928-a740-a18a30c01fa6 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:55.350742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c376c90-8a14-4928-a740-a18a30c01fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:55.351053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c376c90-8a14-4928-a740-a18a30c01fa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:55.352516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c376c90-8a14-4928-a740-a18a30c01fa6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:55.354086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:55.365535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 547/1105] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:11:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:55.371895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:55.419928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:55.470058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a42b39-9b00-431b-a1c5-3141f191bbdb None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:55.503352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f8333-d1aa-4084-a626-c3fa5a232238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:55.504031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8333-d1aa-4084-a626-c3fa5a232238 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:55.504318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8333-d1aa-4084-a626-c3fa5a232238 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:55.504593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8333-d1aa-4084-a626-c3fa5a232238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:55.505431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f8333-d1aa-4084-a626-c3fa5a232238 None admin] RBAC: Adding query filter params (role.id=f88a7c60dc0b4604be9ca139688c9ae7, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=edc938bed535461e974daf01bb99e9a6) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:55.505745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f8333-d1aa-4084-a626-c3fa5a232238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:55.507167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f8333-d1aa-4084-a626-c3fa5a232238 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:55.512920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 559/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:11:55 2022] GET /identity/v3/role_assignments?user.id=edc938bed535461e974daf01bb99e9a6&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=f88a7c60dc0b4604be9ca139688c9ae7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:57.117689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a110b-a119-42b3-80b0-a77991fcbb1c None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:57.118205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a110b-a119-42b3-80b0-a77991fcbb1c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:57.118290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a110b-a119-42b3-80b0-a77991fcbb1c None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:57.119580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 548/1107] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:57.124826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549f33-7690-4f3a-8a2d-83c5b2e74306 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:57.125264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549f33-7690-4f3a-8a2d-83c5b2e74306 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:57.125367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549f33-7690-4f3a-8a2d-83c5b2e74306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:57.143021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9a549f33-7690-4f3a-8a2d-83c5b2e74306 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:57.144228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9a549f33-7690-4f3a-8a2d-83c5b2e74306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:57.208504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 560/1108] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:57.233115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a110b-a119-42b3-80b0-a77991fcbb1c None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:57.329012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a110b-a119-42b3-80b0-a77991fcbb1c None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:57.361769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c1cb0-f3f5-40bd-8cd9-39e0f7f2a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:57.362532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c1cb0-f3f5-40bd-8cd9-39e0f7f2a851 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:57.362813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c1cb0-f3f5-40bd-8cd9-39e0f7f2a851 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:57.363086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c1cb0-f3f5-40bd-8cd9-39e0f7f2a851 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:57.364261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c1cb0-f3f5-40bd-8cd9-39e0f7f2a851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:57.365680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c1cb0-f3f5-40bd-8cd9-39e0f7f2a851 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:57.367237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 549/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:11:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:57.372657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a549f33-7690-4f3a-8a2d-83c5b2e74306 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:57.413968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:57.467835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a549f33-7690-4f3a-8a2d-83c5b2e74306 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:57.500455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fc531-926a-49a4-8922-227c1a697f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:57.501135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fc531-926a-49a4-8922-227c1a697f2a None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:57.501417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fc531-926a-49a4-8922-227c1a697f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:57.501693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fc531-926a-49a4-8922-227c1a697f2a None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:57.502395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fc531-926a-49a4-8922-227c1a697f2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:57.503807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fc531-926a-49a4-8922-227c1a697f2a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:57.522619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 561/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:11:57 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:59.742102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609727e-139d-49cf-8d00-1b09824053b8 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:59.742631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609727e-139d-49cf-8d00-1b09824053b8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:59.742631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609727e-139d-49cf-8d00-1b09824053b8 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:59.744027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 550/1111] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:59.749218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0f85e-d0fa-4f94-b79c-07db280bb817 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:59.749617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0f85e-d0fa-4f94-b79c-07db280bb817 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:59.749786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0f85e-d0fa-4f94-b79c-07db280bb817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:59.767333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-5bf0f85e-d0fa-4f94-b79c-07db280bb817 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:59.768542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5bf0f85e-d0fa-4f94-b79c-07db280bb817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:59.833093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 562/1112] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:11:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:59.856176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609727e-139d-49cf-8d00-1b09824053b8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:59.915786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:59.955381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609727e-139d-49cf-8d00-1b09824053b8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:59.988134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:59.988978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:59.989261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:59.989534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] PATH_INFO: `/v3/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:59.990994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] RBAC: Authorizing `identity:get_role(role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:59.992443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:59.993742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 551/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:11:59 2022] GET /identity/v3/roles/cabb92ea03d84bd399329e57ac452f9c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:59.999508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0f85e-d0fa-4f94-b79c-07db280bb817 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:00.026608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:00.096752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0f85e-d0fa-4f94-b79c-07db280bb817 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:00.131049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:00.131930 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:00.132218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:00.132496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] PATH_INFO: `/v3/users/5d9faf8bfe6047c0828da27ea3430792` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:00.145253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] RBAC: Authorizing `identity:get_user(user_id=5d9faf8bfe6047c0828da27ea3430792)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:00.146675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:00.148343 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 563/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:11:59 2022] GET /identity/v3/users/5d9faf8bfe6047c0828da27ea3430792 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:00.154010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:00.250670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8b2d-a598-4eb0-9a3f-3f35ad3d36ee None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:00.277635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:00.284344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:00.284986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:00.285354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:00.285647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:00.350157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:00.351584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:00.446344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 552/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:12:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:00.452200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:00.534608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:00.548847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7a771-92fe-4d2f-b478-c93ff1680988 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:00.581131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbe649-adad-4e74-9756-3222d196d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:00.581984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbe649-adad-4e74-9756-3222d196d661 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:00.582265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbe649-adad-4e74-9756-3222d196d661 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:00.582538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbe649-adad-4e74-9756-3222d196d661 None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:00.583708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbe649-adad-4e74-9756-3222d196d661 None admin] RBAC: Authorizing `identity:get_project(project_id=ba58af6fbb574b2b85b5129022ebd56f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:00.585185 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbe649-adad-4e74-9756-3222d196d661 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:00.587116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 564/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:00 2022] GET /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:00.593145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:00.622373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:00.689918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a405d0-ec72-4c61-8010-043b61a43b7d None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:00.723589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7a78f-1525-4353-a96f-11415f9902df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:00.724260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a78f-1525-4353-a96f-11415f9902df None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:00.724544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a78f-1525-4353-a96f-11415f9902df None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:00.724824 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a78f-1525-4353-a96f-11415f9902df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:00.725656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7a78f-1525-4353-a96f-11415f9902df None admin] RBAC: Adding query filter params (role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=ba58af6fbb574b2b85b5129022ebd56f, user.id=5d9faf8bfe6047c0828da27ea3430792) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:00.725979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7a78f-1525-4353-a96f-11415f9902df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:00.727396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7a78f-1525-4353-a96f-11415f9902df None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:00.732705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 553/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:12:00 2022] GET /identity/v3/role_assignments?user.id=5d9faf8bfe6047c0828da27ea3430792&scope.project.id=ba58af6fbb574b2b85b5129022ebd56f&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:02.860968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28bb8-62fb-4e67-b254-ad4afa05ddb1 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:02.861502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28bb8-62fb-4e67-b254-ad4afa05ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:02.861502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec28bb8-62fb-4e67-b254-ad4afa05ddb1 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:02.862929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 565/1118] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:02.868084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739df7ba-fe14-4383-9ad3-e764dcf15380 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:02.868523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739df7ba-fe14-4383-9ad3-e764dcf15380 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:02.868616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739df7ba-fe14-4383-9ad3-e764dcf15380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:02.885468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-739df7ba-fe14-4383-9ad3-e764dcf15380 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:02.886628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-739df7ba-fe14-4383-9ad3-e764dcf15380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:02.951619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 554/1119] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:02.974296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28bb8-62fb-4e67-b254-ad4afa05ddb1 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:03.071445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec28bb8-62fb-4e67-b254-ad4afa05ddb1 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:03.103905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:03.104736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:03.105019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:03.105291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] PATH_INFO: `/v3/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:03.106730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] RBAC: Authorizing `identity:get_role(role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:03.108120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:03.109955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 566/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:02 2022] GET /identity/v3/roles/cabb92ea03d84bd399329e57ac452f9c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:03.115437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739df7ba-fe14-4383-9ad3-e764dcf15380 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:03.139219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:03.210671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739df7ba-fe14-4383-9ad3-e764dcf15380 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:03.243336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:03.244220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:03.244502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:03.244775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] PATH_INFO: `/v3/users/5d9faf8bfe6047c0828da27ea3430792` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:03.245936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] RBAC: Authorizing `identity:get_user(user_id=5d9faf8bfe6047c0828da27ea3430792)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:03.247345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:03.248943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 555/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:03 2022] GET /identity/v3/users/5d9faf8bfe6047c0828da27ea3430792 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:03.254423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:03.277544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:03.349891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6251f58-0583-4b6d-9d6b-d28bd6b17bf1 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:03.382461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd488c-58ae-4610-8deb-959837568459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:03.383236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd488c-58ae-4610-8deb-959837568459 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:03.383518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd488c-58ae-4610-8deb-959837568459 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:03.383791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd488c-58ae-4610-8deb-959837568459 None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:03.384970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd488c-58ae-4610-8deb-959837568459 None admin] RBAC: Authorizing `identity:get_project(project_id=ba58af6fbb574b2b85b5129022ebd56f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:03.386376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd488c-58ae-4610-8deb-959837568459 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:03.388041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 567/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:03 2022] GET /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:03.393749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:03.480513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:03.490010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfdd7a-6b9a-4c14-bae7-ac142def6519 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:03.522529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac9c0e-979e-4416-a6f2-85cfd4ca95d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:03.523402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9c0e-979e-4416-a6f2-85cfd4ca95d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:03.523682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9c0e-979e-4416-a6f2-85cfd4ca95d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:03.523986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9c0e-979e-4416-a6f2-85cfd4ca95d6 None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f/users/5d9faf8bfe6047c0828da27ea3430792/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:03.526153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac9c0e-979e-4416-a6f2-85cfd4ca95d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba58af6fbb574b2b85b5129022ebd56f, user_id=5d9faf8bfe6047c0828da27ea3430792, role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:03.527572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac9c0e-979e-4416-a6f2-85cfd4ca95d6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:03.540255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 556/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:12:03 2022] PUT /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f/users/5d9faf8bfe6047c0828da27ea3430792/roles/cabb92ea03d84bd399329e57ac452f9c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:12:05.672998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92416-164b-4f2d-b140-59a4080de920 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:05.673528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92416-164b-4f2d-b140-59a4080de920 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:05.673528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92416-164b-4f2d-b140-59a4080de920 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:05.674910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 568/1124] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:05.680035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d5783-d500-4c47-a51b-c3ee49ae3dd4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:05.680437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d5783-d500-4c47-a51b-c3ee49ae3dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:05.680589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d5783-d500-4c47-a51b-c3ee49ae3dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:05.697424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-552d5783-d500-4c47-a51b-c3ee49ae3dd4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:05.698622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-552d5783-d500-4c47-a51b-c3ee49ae3dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:05.762606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 557/1125] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:05.785789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92416-164b-4f2d-b140-59a4080de920 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:05.874858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:05.885849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92416-164b-4f2d-b140-59a4080de920 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:05.917499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:05.918338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:05.918628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:05.918900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] PATH_INFO: `/v3/roles/cabb92ea03d84bd399329e57ac452f9c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:05.920393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] RBAC: Authorizing `identity:get_role(role_id=cabb92ea03d84bd399329e57ac452f9c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:05.922277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:05.923685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 569/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:05 2022] GET /identity/v3/roles/cabb92ea03d84bd399329e57ac452f9c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:05.929308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d5783-d500-4c47-a51b-c3ee49ae3dd4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:06.007978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:06.026275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d5783-d500-4c47-a51b-c3ee49ae3dd4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:06.059161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:06.060038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:06.060317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:06.060589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] PATH_INFO: `/v3/users/5d9faf8bfe6047c0828da27ea3430792` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:06.061749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] RBAC: Authorizing `identity:get_user(user_id=5d9faf8bfe6047c0828da27ea3430792)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:06.063725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:06.065330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 558/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:05 2022] GET /identity/v3/users/5d9faf8bfe6047c0828da27ea3430792 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:06.070960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:06.165707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509cea8-b40b-47fe-a09c-f3a9c6c3889a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:06.197879 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:06.198443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:06.198724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:06.198999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:06.224692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:06.263558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:06.265003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:06.357647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 570/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:12:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:06.363959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:06.458404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18ab4d-23ad-4030-b319-7cfa52b12628 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:06.491446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fbdb7-1f35-4191-b80f-2946ac54ea2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:06.492305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fbdb7-1f35-4191-b80f-2946ac54ea2a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:06.492586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fbdb7-1f35-4191-b80f-2946ac54ea2a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:06.492857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fbdb7-1f35-4191-b80f-2946ac54ea2a None admin] PATH_INFO: `/v3/projects/ba58af6fbb574b2b85b5129022ebd56f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:06.494005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fbdb7-1f35-4191-b80f-2946ac54ea2a None admin] RBAC: Authorizing `identity:get_project(project_id=ba58af6fbb574b2b85b5129022ebd56f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:06.495418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fbdb7-1f35-4191-b80f-2946ac54ea2a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:06.497075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 559/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:06 2022] GET /identity/v3/projects/ba58af6fbb574b2b85b5129022ebd56f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:06.502816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:06.557975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:06.598036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d0d9-b661-45fc-b637-70b5d8530afa None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:06.629751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d726439-15b8-4bb5-937a-4a0a62cdb36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:06.630378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d726439-15b8-4bb5-937a-4a0a62cdb36b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:06.630661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d726439-15b8-4bb5-937a-4a0a62cdb36b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:06.630935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d726439-15b8-4bb5-937a-4a0a62cdb36b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:06.631769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d726439-15b8-4bb5-937a-4a0a62cdb36b None admin] RBAC: Adding query filter params (role.id=cabb92ea03d84bd399329e57ac452f9c, scope.project.id=ba58af6fbb574b2b85b5129022ebd56f, user.id=5d9faf8bfe6047c0828da27ea3430792) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:06.632110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d726439-15b8-4bb5-937a-4a0a62cdb36b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:06.633526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d726439-15b8-4bb5-937a-4a0a62cdb36b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:06.639102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 571/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:12:06 2022] GET /identity/v3/role_assignments?user.id=5d9faf8bfe6047c0828da27ea3430792&scope.project.id=ba58af6fbb574b2b85b5129022ebd56f&role.id=cabb92ea03d84bd399329e57ac452f9c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:08.823209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44552d8-c1a6-4c3c-b16d-d56cc0c40c86 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:08.823742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44552d8-c1a6-4c3c-b16d-d56cc0c40c86 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:08.823742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44552d8-c1a6-4c3c-b16d-d56cc0c40c86 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:08.825155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 560/1131] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:08.830462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8cbe-9372-4e37-8ee8-0da8716e2075 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:08.830833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8cbe-9372-4e37-8ee8-0da8716e2075 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:08.831022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8cbe-9372-4e37-8ee8-0da8716e2075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:08.848443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c02e8cbe-9372-4e37-8ee8-0da8716e2075 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:08.849631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c02e8cbe-9372-4e37-8ee8-0da8716e2075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:08.914836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 572/1132] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:08.939926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44552d8-c1a6-4c3c-b16d-d56cc0c40c86 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:08.958476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:09.037510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44552d8-c1a6-4c3c-b16d-d56cc0c40c86 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:09.070226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903a6ad-05f5-444b-b750-f9ec23d04233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:09.071001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903a6ad-05f5-444b-b750-f9ec23d04233 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:09.071286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903a6ad-05f5-444b-b750-f9ec23d04233 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:09.071564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903a6ad-05f5-444b-b750-f9ec23d04233 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:09.072752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903a6ad-05f5-444b-b750-f9ec23d04233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:09.074217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903a6ad-05f5-444b-b750-f9ec23d04233 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:09.075744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 561/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:12:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:09.081647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8cbe-9372-4e37-8ee8-0da8716e2075 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:09.132719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:09.180589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8cbe-9372-4e37-8ee8-0da8716e2075 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:09.221381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc6b3a-13ef-4ec8-9b8b-ea42a6edf4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:09.222055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc6b3a-13ef-4ec8-9b8b-ea42a6edf4ea None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:09.222322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc6b3a-13ef-4ec8-9b8b-ea42a6edf4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:09.222596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc6b3a-13ef-4ec8-9b8b-ea42a6edf4ea None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:09.223320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc6b3a-13ef-4ec8-9b8b-ea42a6edf4ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:09.224777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc6b3a-13ef-4ec8-9b8b-ea42a6edf4ea None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:09.249371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 573/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:12:09 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:12:11.458145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5825f19-5f73-4df4-b487-5f32a51badf5 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:11.458709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5825f19-5f73-4df4-b487-5f32a51badf5 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:11.458709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5825f19-5f73-4df4-b487-5f32a51badf5 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:11.460340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 562/1135] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:11.465449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c24977-d7ce-415f-a84f-e9a1d23f2a14 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:11.465817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c24977-d7ce-415f-a84f-e9a1d23f2a14 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:11.466015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c24977-d7ce-415f-a84f-e9a1d23f2a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:11.482781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f0c24977-d7ce-415f-a84f-e9a1d23f2a14 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:11.484583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f0c24977-d7ce-415f-a84f-e9a1d23f2a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:11.548471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 574/1136] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:11.572410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5825f19-5f73-4df4-b487-5f32a51badf5 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:11.653820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:11.669691 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5825f19-5f73-4df4-b487-5f32a51badf5 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:11.701865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26ab5a-b012-45ac-87a4-27889d8ae1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:11.702655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26ab5a-b012-45ac-87a4-27889d8ae1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:11.702936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26ab5a-b012-45ac-87a4-27889d8ae1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:11.703207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26ab5a-b012-45ac-87a4-27889d8ae1a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:11.704393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26ab5a-b012-45ac-87a4-27889d8ae1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:11.705812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26ab5a-b012-45ac-87a4-27889d8ae1a7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:11.707370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 563/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:12:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:11.713174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c24977-d7ce-415f-a84f-e9a1d23f2a14 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:11.746024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:11.809453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c24977-d7ce-415f-a84f-e9a1d23f2a14 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:11.842003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e62c87-5d0d-4839-a798-acdfa7e5d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:11.842655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e62c87-5d0d-4839-a798-acdfa7e5d1d3 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:11.842943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e62c87-5d0d-4839-a798-acdfa7e5d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:11.843758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e62c87-5d0d-4839-a798-acdfa7e5d1d3 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:11.844503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e62c87-5d0d-4839-a798-acdfa7e5d1d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:11.845919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e62c87-5d0d-4839-a798-acdfa7e5d1d3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:11.864717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 575/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:12:11 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:12:14.097629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3a3c-eff4-45c0-a7d2-e97825e8eafc None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:14.098163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3a3c-eff4-45c0-a7d2-e97825e8eafc None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:14.098163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3a3c-eff4-45c0-a7d2-e97825e8eafc None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:14.099538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 564/1139] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:14.104846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a6433-d2b6-487d-998e-522021f0d445 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:14.105247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a6433-d2b6-487d-998e-522021f0d445 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:14.105409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a6433-d2b6-487d-998e-522021f0d445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:14.122238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a19a6433-d2b6-487d-998e-522021f0d445 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:14.123411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a19a6433-d2b6-487d-998e-522021f0d445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:14.187860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 576/1140] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:14.210724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3a3c-eff4-45c0-a7d2-e97825e8eafc None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:14.308845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3a3c-eff4-45c0-a7d2-e97825e8eafc None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:14.341638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:14.342415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:14.342703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:14.343005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:14.352961 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:14.361129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:14.362223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:12:14.363262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 565/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:12:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:14.368035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a6433-d2b6-487d-998e-522021f0d445 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:14.388809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:14.463580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a6433-d2b6-487d-998e-522021f0d445 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:14.496395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:14.497047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:14.497332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:14.497609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:14.498358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:14.498666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:14.500075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:14.507775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 577/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:12:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:14.513302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:14.528964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:14.610458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613baaa4-eed4-4bfe-a0e8-da7e5ee4d415 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:14.641895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:14.642750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:14.643031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:14.643305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] PATH_INFO: `/v3/users/43fefa71fe7f4a459283c2dedad95707` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:14.656351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] RBAC: Authorizing `identity:get_user(user_id=43fefa71fe7f4a459283c2dedad95707)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:14.657795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:14.659393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 566/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:14 2022] GET /identity/v3/users/43fefa71fe7f4a459283c2dedad95707 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:14.664934 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:14.745735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:14.763268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff799-7081-4c2b-beb1-399136a000c3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:14.795717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:14.796400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:14.796686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:14.796966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:14.861066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:14.862496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:14.940780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:14.960603 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 578/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:12:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:14.966438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:15.034600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:15.062216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429fdd2-c05f-43b8-9513-a8bf5f5b11bf None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:15.095413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b72f21-0cdd-4ff7-8632-1d87741dbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:15.096271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72f21-0cdd-4ff7-8632-1d87741dbce6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:15.096554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72f21-0cdd-4ff7-8632-1d87741dbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:15.096827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72f21-0cdd-4ff7-8632-1d87741dbce6 None admin] PATH_INFO: `/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:15.097992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b72f21-0cdd-4ff7-8632-1d87741dbce6 None admin] RBAC: Authorizing `identity:get_project(project_id=7dd32d5a0fdb40389b14b3a395b28dce)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:15.099429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b72f21-0cdd-4ff7-8632-1d87741dbce6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:15.101126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 567/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:14 2022] GET /identity/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:15.107038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:15.201741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6f8e2-d25e-4050-ba5e-7efe110f5f4e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:15.234403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c137ea8-11d8-40f7-98c9-bfbf86573842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:15.235039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137ea8-11d8-40f7-98c9-bfbf86573842 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:15.235326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137ea8-11d8-40f7-98c9-bfbf86573842 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:15.235601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137ea8-11d8-40f7-98c9-bfbf86573842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:15.236465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c137ea8-11d8-40f7-98c9-bfbf86573842 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=7dd32d5a0fdb40389b14b3a395b28dce, user.id=43fefa71fe7f4a459283c2dedad95707) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:15.236779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c137ea8-11d8-40f7-98c9-bfbf86573842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:15.238194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c137ea8-11d8-40f7-98c9-bfbf86573842 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:15.243396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 579/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:12:15 2022] GET /identity/v3/role_assignments?user.id=43fefa71fe7f4a459283c2dedad95707&scope.project.id=7dd32d5a0fdb40389b14b3a395b28dce&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:17.367000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6af5ce-0bef-4284-a1fe-0750fc1e44ea None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:17.367510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6af5ce-0bef-4284-a1fe-0750fc1e44ea None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:17.367629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6af5ce-0bef-4284-a1fe-0750fc1e44ea None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:17.368956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 568/1147] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:17.374671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aabb8f-3be0-430a-a9c7-f6efc51df96b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:17.375088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aabb8f-3be0-430a-a9c7-f6efc51df96b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:17.375228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aabb8f-3be0-430a-a9c7-f6efc51df96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:17.391422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-44aabb8f-3be0-430a-a9c7-f6efc51df96b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:17.392615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-44aabb8f-3be0-430a-a9c7-f6efc51df96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:17.433475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:17.457264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 580/1148] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:17.480004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6af5ce-0bef-4284-a1fe-0750fc1e44ea None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:17.578306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6af5ce-0bef-4284-a1fe-0750fc1e44ea None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:17.610352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:17.611120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:17.611398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:17.611667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:17.618916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:17.622097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:17.630589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:17.631694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:12:17.632916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 569/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:17.637705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aabb8f-3be0-430a-a9c7-f6efc51df96b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:17.732114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aabb8f-3be0-430a-a9c7-f6efc51df96b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:17.764234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:17.765438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:17.765719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:17.765997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:17.766737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:17.767042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:17.768422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:17.776193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 581/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:12:17.781940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:17.879084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f4102-420d-4419-b0c1-8cf39212b085 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:17.910406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:17.911255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:17.911536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:17.911811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] PATH_INFO: `/v3/users/43fefa71fe7f4a459283c2dedad95707` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:17.913013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] RBAC: Authorizing `identity:get_user(user_id=43fefa71fe7f4a459283c2dedad95707)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:17.914439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:17.916061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 570/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:17 2022] GET /identity/v3/users/43fefa71fe7f4a459283c2dedad95707 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:17.922068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:17.960728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:18.018773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dd8c0-4a5a-4609-833a-768316455791 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:18.050420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2c2e-54be-47a4-9737-687dae4c24d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:18.051253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2c2e-54be-47a4-9737-687dae4c24d1 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:18.051532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2c2e-54be-47a4-9737-687dae4c24d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:18.051802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2c2e-54be-47a4-9737-687dae4c24d1 None admin] PATH_INFO: `/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:18.052995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca2c2e-54be-47a4-9737-687dae4c24d1 None admin] RBAC: Authorizing `identity:get_project(project_id=7dd32d5a0fdb40389b14b3a395b28dce)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:18.054410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca2c2e-54be-47a4-9737-687dae4c24d1 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:18.056090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 582/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:17 2022] GET /identity/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:18.062521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:18.126786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:18.159216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe1eb-1d87-44ef-8c2e-849769b6d2ee None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:18.190892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40eae0-19f5-404a-bc50-bfb854c809a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:18.191760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40eae0-19f5-404a-bc50-bfb854c809a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:18.192067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40eae0-19f5-404a-bc50-bfb854c809a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:18.192340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40eae0-19f5-404a-bc50-bfb854c809a7 None admin] PATH_INFO: `/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce/users/43fefa71fe7f4a459283c2dedad95707/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:18.194299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40eae0-19f5-404a-bc50-bfb854c809a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dd32d5a0fdb40389b14b3a395b28dce, user_id=43fefa71fe7f4a459283c2dedad95707, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:18.195713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40eae0-19f5-404a-bc50-bfb854c809a7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:18.207456 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 571/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:12:18 2022] PUT /identity/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce/users/43fefa71fe7f4a459283c2dedad95707/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:12:20.346220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec582fc1-6dc4-427a-bf87-8caa871ff8c5 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:20.346730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec582fc1-6dc4-427a-bf87-8caa871ff8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:20.346840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec582fc1-6dc4-427a-bf87-8caa871ff8c5 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:20.348198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 583/1154] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:20.353541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78f3c-808d-458b-8d1d-88cf99da8941 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:20.353984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78f3c-808d-458b-8d1d-88cf99da8941 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:20.354104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78f3c-808d-458b-8d1d-88cf99da8941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:20.370428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0bd78f3c-808d-458b-8d1d-88cf99da8941 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:20.371595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0bd78f3c-808d-458b-8d1d-88cf99da8941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:20.435143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 572/1155] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:20.458739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec582fc1-6dc4-427a-bf87-8caa871ff8c5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:20.556676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec582fc1-6dc4-427a-bf87-8caa871ff8c5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:20.563830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:20.591162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:20.592420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:20.592868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:20.593297 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:20.604975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:20.612554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:20.613636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:12:20.614649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 584/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:12:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:20.619634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd78f3c-808d-458b-8d1d-88cf99da8941 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:20.675227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:20.716170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd78f3c-808d-458b-8d1d-88cf99da8941 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:20.750401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:20.751051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:20.751335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:20.751609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:20.752386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:20.752693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:20.754056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:20.761986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 573/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:12:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:20.768481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:20.865928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6114f7-df16-404b-863c-4d920cd1deb6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:20.898798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:20.899660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:20.899976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:20.900257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] PATH_INFO: `/v3/users/43fefa71fe7f4a459283c2dedad95707` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:20.901432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] RBAC: Authorizing `identity:get_user(user_id=43fefa71fe7f4a459283c2dedad95707)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:20.902864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:20.904523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 585/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:20 2022] GET /identity/v3/users/43fefa71fe7f4a459283c2dedad95707 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:20.910286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:21.009202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c90e92-50db-4819-8aa4-3bd82966f9e5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:21.027650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:21.041524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:21.042168 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:21.042451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:21.042728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:21.106759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:21.108759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:21.202058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 574/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:12:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:21.207900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:21.243325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:21.306945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8176-ff01-43b5-a832-51efc28f7b5a None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:21.338683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b98ce-b315-47fb-9129-d58fb14180c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:21.339528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b98ce-b315-47fb-9129-d58fb14180c6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:21.339847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b98ce-b315-47fb-9129-d58fb14180c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:21.340128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b98ce-b315-47fb-9129-d58fb14180c6 None admin] PATH_INFO: `/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:21.341298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b98ce-b315-47fb-9129-d58fb14180c6 None admin] RBAC: Authorizing `identity:get_project(project_id=7dd32d5a0fdb40389b14b3a395b28dce)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:21.343224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b98ce-b315-47fb-9129-d58fb14180c6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:21.344937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 586/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:21 2022] GET /identity/v3/projects/7dd32d5a0fdb40389b14b3a395b28dce => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:21.351003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:21.371936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:21.445772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988882e9-de2d-48cd-b239-dca1c9b9285a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:21.478479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de8610-e170-47c4-9309-1fd1a4ee1aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:21.479080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8610-e170-47c4-9309-1fd1a4ee1aad None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:21.479372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8610-e170-47c4-9309-1fd1a4ee1aad None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:21.479662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8610-e170-47c4-9309-1fd1a4ee1aad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:21.480530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de8610-e170-47c4-9309-1fd1a4ee1aad None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=7dd32d5a0fdb40389b14b3a395b28dce, user.id=43fefa71fe7f4a459283c2dedad95707) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:21.480847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de8610-e170-47c4-9309-1fd1a4ee1aad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:21.482268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de8610-e170-47c4-9309-1fd1a4ee1aad None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:21.488584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 575/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:12:21 2022] GET /identity/v3/role_assignments?user.id=43fefa71fe7f4a459283c2dedad95707&scope.project.id=7dd32d5a0fdb40389b14b3a395b28dce&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:23.666973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea93a17-b64b-46e9-9023-a1b3483f19c1 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:23.667532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea93a17-b64b-46e9-9023-a1b3483f19c1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:23.667532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea93a17-b64b-46e9-9023-a1b3483f19c1 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:23.668960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 587/1162] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:23.674151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34fe96-03f9-462e-8a4d-f63782d99fb1 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:23.674569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34fe96-03f9-462e-8a4d-f63782d99fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:23.674709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34fe96-03f9-462e-8a4d-f63782d99fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:23.691511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-7c34fe96-03f9-462e-8a4d-f63782d99fb1 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:23.692709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7c34fe96-03f9-462e-8a4d-f63782d99fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:23.756819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 576/1163] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:12:23.779005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea93a17-b64b-46e9-9023-a1b3483f19c1 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:23.877595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea93a17-b64b-46e9-9023-a1b3483f19c1 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:23.879067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:23.910228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:23.910860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:23.911146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:23.911424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:23.977853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:23.979288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:24.073345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 588/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:12:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:24.079377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34fe96-03f9-462e-8a4d-f63782d99fb1 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:24.094031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:24.174875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34fe96-03f9-462e-8a4d-f63782d99fb1 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:24.206848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f20f71-7267-48ea-b5d7-deedb1d426ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:24.207583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f20f71-7267-48ea-b5d7-deedb1d426ae None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:24.207890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f20f71-7267-48ea-b5d7-deedb1d426ae None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:24.208170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f20f71-7267-48ea-b5d7-deedb1d426ae None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:24.214882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f20f71-7267-48ea-b5d7-deedb1d426ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:24.216344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f20f71-7267-48ea-b5d7-deedb1d426ae None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-98f20f71-7267-48ea-b5d7-deedb1d426ae None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:24.230566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:24.231840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:12:24.233479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 577/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:12:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:24.238189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:24.240371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:24.350064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e44913-ff8b-442c-bba1-247207efd6cf None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:24.382626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5aa189-337a-465a-a0e1-b38d8699bd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:24.383280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aa189-337a-465a-a0e1-b38d8699bd59 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:24.383564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aa189-337a-465a-a0e1-b38d8699bd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:24.383861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aa189-337a-465a-a0e1-b38d8699bd59 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:24.384586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aa189-337a-465a-a0e1-b38d8699bd59 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:24.384895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aa189-337a-465a-a0e1-b38d8699bd59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:24.386270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aa189-337a-465a-a0e1-b38d8699bd59 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:24.393211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 589/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Fri May 27 03:12:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:26.546978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e4d97-c814-49d3-9498-1e96bef5ddbf None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:26.547544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e4d97-c814-49d3-9498-1e96bef5ddbf None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:26.547544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e4d97-c814-49d3-9498-1e96bef5ddbf None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:26.548909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 578/1167] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:26.554203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271c2c2-f208-45be-bb63-1d03246a1934 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:26.554571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271c2c2-f208-45be-bb63-1d03246a1934 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:26.554766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271c2c2-f208-45be-bb63-1d03246a1934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:26.571719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4271c2c2-f208-45be-bb63-1d03246a1934 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:26.572913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4271c2c2-f208-45be-bb63-1d03246a1934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:26.629783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:26.637527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 590/1168] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:26.660587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e4d97-c814-49d3-9498-1e96bef5ddbf None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:26.734629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:26.759590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e4d97-c814-49d3-9498-1e96bef5ddbf None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:26.791216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223e0df-769d-4f26-bafd-fa875c95cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:26.791876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223e0df-769d-4f26-bafd-fa875c95cc1d None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:26.792160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223e0df-769d-4f26-bafd-fa875c95cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:26.792432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223e0df-769d-4f26-bafd-fa875c95cc1d None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:26.793170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223e0df-769d-4f26-bafd-fa875c95cc1d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:26.794521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223e0df-769d-4f26-bafd-fa875c95cc1d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:26.808449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 579/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Fri May 27 03:12:26 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:12:28.992593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9940138-a74e-4bdb-8703-985aa152336d None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:28.993053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9940138-a74e-4bdb-8703-985aa152336d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:28.993158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9940138-a74e-4bdb-8703-985aa152336d None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:28.994521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 591/1170] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:28.999956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef852bd-88fc-4d7e-a128-4878b55ab82b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:29.000360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef852bd-88fc-4d7e-a128-4878b55ab82b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:29.000527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef852bd-88fc-4d7e-a128-4878b55ab82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:29.023326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2ef852bd-88fc-4d7e-a128-4878b55ab82b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:29.024538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2ef852bd-88fc-4d7e-a128-4878b55ab82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:29.088852 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 580/1171] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:29.112955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9940138-a74e-4bdb-8703-985aa152336d None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:29.210929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9940138-a74e-4bdb-8703-985aa152336d None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:29.243458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5e647-c290-44fe-9ddc-ad515c82f1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:29.244338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5e647-c290-44fe-9ddc-ad515c82f1eb None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:29.244623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5e647-c290-44fe-9ddc-ad515c82f1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:29.244901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5e647-c290-44fe-9ddc-ad515c82f1eb None admin] PATH_INFO: `/v3/domains/b969b2de17d847db89784629222265c0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:29.254977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5e647-c290-44fe-9ddc-ad515c82f1eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=b969b2de17d847db89784629222265c0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:29.256445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5e647-c290-44fe-9ddc-ad515c82f1eb None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:29.258052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 592/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Fri May 27 03:12:29 2022] GET /identity/v3/domains/b969b2de17d847db89784629222265c0 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:29.264353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef852bd-88fc-4d7e-a128-4878b55ab82b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:29.323267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:29.360839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef852bd-88fc-4d7e-a128-4878b55ab82b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:29.393470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7557ca-df1e-4bf4-8792-a62329ac66cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:29.394125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7557ca-df1e-4bf4-8792-a62329ac66cb None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:29.394412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7557ca-df1e-4bf4-8792-a62329ac66cb None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:29.394693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7557ca-df1e-4bf4-8792-a62329ac66cb None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:29.395403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7557ca-df1e-4bf4-8792-a62329ac66cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:29.396864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7557ca-df1e-4bf4-8792-a62329ac66cb None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:29.421175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 581/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:12:29 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:12:31.623128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97b64-1756-4f9a-b3d8-b0a3f5bbcb02 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:31.623689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97b64-1756-4f9a-b3d8-b0a3f5bbcb02 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:31.623689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97b64-1756-4f9a-b3d8-b0a3f5bbcb02 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:31.625090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 593/1174] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:31.630217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86ce13-4116-4ba4-be69-acce3d339964 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:31.630595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86ce13-4116-4ba4-be69-acce3d339964 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:31.630776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86ce13-4116-4ba4-be69-acce3d339964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:31.647193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-eb86ce13-4116-4ba4-be69-acce3d339964 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:31.648381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-eb86ce13-4116-4ba4-be69-acce3d339964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:31.712098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 582/1175] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:31.737213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e97b64-1756-4f9a-b3d8-b0a3f5bbcb02 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:31.791493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:31.834776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e97b64-1756-4f9a-b3d8-b0a3f5bbcb02 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:31.867215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0d985-2a7e-415c-9081-f361cb3bb322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:31.868082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d985-2a7e-415c-9081-f361cb3bb322 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:31.868369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d985-2a7e-415c-9081-f361cb3bb322 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:31.868647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d985-2a7e-415c-9081-f361cb3bb322 None admin] PATH_INFO: `/v3/domains/b969b2de17d847db89784629222265c0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:31.869812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0d985-2a7e-415c-9081-f361cb3bb322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b969b2de17d847db89784629222265c0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:31.871239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0d985-2a7e-415c-9081-f361cb3bb322 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:31.872891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 594/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Fri May 27 03:12:31 2022] GET /identity/v3/domains/b969b2de17d847db89784629222265c0 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:31.878424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86ce13-4116-4ba4-be69-acce3d339964 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:31.923030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:31.974720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86ce13-4116-4ba4-be69-acce3d339964 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:32.007151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152bd70-cecb-466c-8da7-13c7311d87a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:32.007810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152bd70-cecb-466c-8da7-13c7311d87a9 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:32.008123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152bd70-cecb-466c-8da7-13c7311d87a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:32.008404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152bd70-cecb-466c-8da7-13c7311d87a9 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:32.009123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152bd70-cecb-466c-8da7-13c7311d87a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:32.010550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152bd70-cecb-466c-8da7-13c7311d87a9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:32.030116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 583/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:12:31 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:12:34.244967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839dcee-7b27-4ad6-8db0-a3dd3af372f2 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:34.245491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839dcee-7b27-4ad6-8db0-a3dd3af372f2 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:34.245491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839dcee-7b27-4ad6-8db0-a3dd3af372f2 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:34.246797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 595/1178] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:34.252451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18db7-e867-4eab-9728-03179b712fcb None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:34.252907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18db7-e867-4eab-9728-03179b712fcb None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:34.253002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18db7-e867-4eab-9728-03179b712fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:34.269814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4fb18db7-e867-4eab-9728-03179b712fcb None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:34.270979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4fb18db7-e867-4eab-9728-03179b712fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:34.335036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 584/1179] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:34.358682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839dcee-7b27-4ad6-8db0-a3dd3af372f2 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:34.464340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839dcee-7b27-4ad6-8db0-a3dd3af372f2 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:34.486443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:34.497153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:34.498435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:34.498718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:34.498994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:34.509438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:34.517025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:34.518117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:12:34.519189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 596/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:34.524344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb18db7-e867-4eab-9728-03179b712fcb None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:34.553665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:34.619570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb18db7-e867-4eab-9728-03179b712fcb None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:34.652632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:34.653282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:34.654057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:34.654330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:34.655076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:34.655382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:34.656780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:34.664510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 585/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:34.670636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:34.769594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f9df9-1f34-454c-adea-b59e6860e0c4 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:34.802975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:34.803845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:34.804136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:34.804411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] PATH_INFO: `/v3/users/e1a109f066bd447e981e3997005d90df` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:34.816868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] RBAC: Authorizing `identity:get_user(user_id=e1a109f066bd447e981e3997005d90df)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:34.818303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:34.819938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 597/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:34 2022] GET /identity/v3/users/e1a109f066bd447e981e3997005d90df => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:34.825817 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:34.916404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:34.923967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96501e-98b9-43a9-bebf-f74c8c67b760 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:34.956875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:34.957520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:34.957804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:34.958080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:35.023329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:35.024853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:35.113312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:35.120884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 586/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:12:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:35.126787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:35.169059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:35.222992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388937-1764-4993-aa3e-c1a907498e9e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:35.254586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa8312-d847-4638-8111-c61ed7867f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:35.255424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa8312-d847-4638-8111-c61ed7867f3f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:35.255706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa8312-d847-4638-8111-c61ed7867f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:35.256010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa8312-d847-4638-8111-c61ed7867f3f None admin] PATH_INFO: `/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:35.257164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa8312-d847-4638-8111-c61ed7867f3f None admin] RBAC: Authorizing `identity:get_project(project_id=8ecbb2573c924a4c9a1b62138a9b46a4)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:35.258583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa8312-d847-4638-8111-c61ed7867f3f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:35.260274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 598/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:35 2022] GET /identity/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:35.266616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:35.362201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aefbd3-81fd-481e-8521-b3496f058b32 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:35.394595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a8534-b711-4da7-9cff-02e005ccf9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:35.395232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a8534-b711-4da7-9cff-02e005ccf9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:35.395512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a8534-b711-4da7-9cff-02e005ccf9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:35.395790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a8534-b711-4da7-9cff-02e005ccf9a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:35.396641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a8534-b711-4da7-9cff-02e005ccf9a0 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=8ecbb2573c924a4c9a1b62138a9b46a4, user.id=e1a109f066bd447e981e3997005d90df) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:35.396960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a8534-b711-4da7-9cff-02e005ccf9a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:35.398366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a8534-b711-4da7-9cff-02e005ccf9a0 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:35.404228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 587/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:12:35 2022] GET /identity/v3/role_assignments?user.id=e1a109f066bd447e981e3997005d90df&scope.project.id=8ecbb2573c924a4c9a1b62138a9b46a4&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:37.542168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93853-3702-4e7b-b732-7429f317d459 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:37.542718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93853-3702-4e7b-b732-7429f317d459 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:37.542718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93853-3702-4e7b-b732-7429f317d459 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:37.544636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 599/1186] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:37.549841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf15cc8-0a88-4ea1-b4e1-096565a1463b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:37.550249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf15cc8-0a88-4ea1-b4e1-096565a1463b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:37.550406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf15cc8-0a88-4ea1-b4e1-096565a1463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:37.567333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0cf15cc8-0a88-4ea1-b4e1-096565a1463b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:37.568534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0cf15cc8-0a88-4ea1-b4e1-096565a1463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:37.619444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:37.633598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 588/1187] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:37.656270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93853-3702-4e7b-b732-7429f317d459 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:37.755023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93853-3702-4e7b-b732-7429f317d459 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:37.765477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:37.789163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:37.789948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:37.790234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:37.790512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:37.801055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:37.808604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:37.809737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:12:37.810658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 600/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:37.815771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf15cc8-0a88-4ea1-b4e1-096565a1463b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:37.911213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf15cc8-0a88-4ea1-b4e1-096565a1463b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:37.944173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:37.944835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:37.945116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:37.945389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:37.946141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:37.946449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:37.947828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:37.955654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 589/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:37.961760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:38.059096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8f29a-493a-4280-a9d8-a12e0ab039f6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:38.095692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:38.096586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:38.096868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:38.097141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] PATH_INFO: `/v3/users/e1a109f066bd447e981e3997005d90df` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:38.098307 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] RBAC: Authorizing `identity:get_user(user_id=e1a109f066bd447e981e3997005d90df)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:38.099726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:38.101373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 601/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:37 2022] GET /identity/v3/users/e1a109f066bd447e981e3997005d90df => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:38.107178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:38.156758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:38.204524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06fb47-9eec-4095-b43e-5461607c9dff None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:38.236886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de4f11-5d83-4e9e-a8de-f72d52a8df31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:38.237738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4f11-5d83-4e9e-a8de-f72d52a8df31 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:38.238022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4f11-5d83-4e9e-a8de-f72d52a8df31 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:38.238297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4f11-5d83-4e9e-a8de-f72d52a8df31 None admin] PATH_INFO: `/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:38.239459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de4f11-5d83-4e9e-a8de-f72d52a8df31 None admin] RBAC: Authorizing `identity:get_project(project_id=8ecbb2573c924a4c9a1b62138a9b46a4)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:38.240918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de4f11-5d83-4e9e-a8de-f72d52a8df31 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:38.242618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 590/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:38 2022] GET /identity/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:38.248557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:38.287525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:38.345459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316e29-6cf4-435d-8891-188ed39ed7bd None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:38.377779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d63c4-2ee9-4b70-bf9e-9ab4b77ecbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:38.378648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d63c4-2ee9-4b70-bf9e-9ab4b77ecbd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:38.378926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d63c4-2ee9-4b70-bf9e-9ab4b77ecbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:38.379195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d63c4-2ee9-4b70-bf9e-9ab4b77ecbd4 None admin] PATH_INFO: `/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4/users/e1a109f066bd447e981e3997005d90df/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:38.381142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d63c4-2ee9-4b70-bf9e-9ab4b77ecbd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ecbb2573c924a4c9a1b62138a9b46a4, user_id=e1a109f066bd447e981e3997005d90df, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:38.382566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d63c4-2ee9-4b70-bf9e-9ab4b77ecbd4 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:38.395586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 602/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:12:38 2022] PUT /identity/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4/users/e1a109f066bd447e981e3997005d90df/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:12:40.533860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3507f-073a-4ec1-b9f0-c45ac24bc6b2 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:40.534390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3507f-073a-4ec1-b9f0-c45ac24bc6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:40.534390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3507f-073a-4ec1-b9f0-c45ac24bc6b2 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:40.535758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 591/1193] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:40.540909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426787d-6113-4566-8277-894508706080 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:40.541278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426787d-6113-4566-8277-894508706080 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:40.541482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426787d-6113-4566-8277-894508706080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:40.557909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0426787d-6113-4566-8277-894508706080 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:40.559073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0426787d-6113-4566-8277-894508706080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:40.624121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 603/1194] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:12:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:40.647225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3507f-073a-4ec1-b9f0-c45ac24bc6b2 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:40.745028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3507f-073a-4ec1-b9f0-c45ac24bc6b2 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:40.761471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:40.778204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:40.778978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:40.779259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:40.779533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:40.789607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:40.797096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:40.798178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:12:40.799073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 592/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:12:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:40.804054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426787d-6113-4566-8277-894508706080 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:40.831393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:40.900993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426787d-6113-4566-8277-894508706080 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:40.933053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:40.933711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:40.933994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:40.934789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:40.935542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:40.935939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:40.937323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:40.945216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 604/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:40.951400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:41.046808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61addb4-08a2-4217-b731-34647dc036f7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:41.078912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:41.079762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:41.080159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:41.080430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] PATH_INFO: `/v3/users/e1a109f066bd447e981e3997005d90df` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:41.081672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] RBAC: Authorizing `identity:get_user(user_id=e1a109f066bd447e981e3997005d90df)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:41.083351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:41.084970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 593/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Fri May 27 03:12:40 2022] GET /identity/v3/users/e1a109f066bd447e981e3997005d90df => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:41.090552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:41.179082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:41.188944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bc701-8a7f-453c-b18b-f923c7904bcf None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:41.221477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:41.222114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:41.222400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:41.222679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:41.286869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:41.288330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:41.373286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:41.383007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 605/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:12:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:41.389076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:41.434326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:41.491261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786b2f-66c6-49e8-af8e-a0838b22b93f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:41.523205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534c810-a681-439d-8e39-34ff78cd7308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:41.524085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c810-a681-439d-8e39-34ff78cd7308 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:41.524369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c810-a681-439d-8e39-34ff78cd7308 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:41.524647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c810-a681-439d-8e39-34ff78cd7308 None admin] PATH_INFO: `/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:41.525863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534c810-a681-439d-8e39-34ff78cd7308 None admin] RBAC: Authorizing `identity:get_project(project_id=8ecbb2573c924a4c9a1b62138a9b46a4)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:41.527292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534c810-a681-439d-8e39-34ff78cd7308 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:41.529553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 594/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Fri May 27 03:12:41 2022] GET /identity/v3/projects/8ecbb2573c924a4c9a1b62138a9b46a4 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:41.535749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:41.630142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b83f9-3437-46f1-8afe-e2dd25e6d566 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:41.662446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c64ed-0527-45f4-a051-bf87e5bb2de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:41.663082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c64ed-0527-45f4-a051-bf87e5bb2de5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:41.663367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c64ed-0527-45f4-a051-bf87e5bb2de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:41.663649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c64ed-0527-45f4-a051-bf87e5bb2de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:41.664512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c64ed-0527-45f4-a051-bf87e5bb2de5 None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=8ecbb2573c924a4c9a1b62138a9b46a4, user.id=e1a109f066bd447e981e3997005d90df) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:41.664828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c64ed-0527-45f4-a051-bf87e5bb2de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:41.666250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c64ed-0527-45f4-a051-bf87e5bb2de5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:41.672475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 606/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:12:41 2022] GET /identity/v3/role_assignments?user.id=e1a109f066bd447e981e3997005d90df&scope.project.id=8ecbb2573c924a4c9a1b62138a9b46a4&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:51.519136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=43 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:51.669911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=73 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:41.341360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c1eeb-3ac5-4506-b2b3-ad67ba562881 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:41.342000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c1eeb-3ac5-4506-b2b3-ad67ba562881 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:41.342000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c1eeb-3ac5-4506-b2b3-ad67ba562881 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:41.343981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 595/1201] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:41.350426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca719d-9fa0-4b6d-af47-e6b262bea8d1 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:41.351019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca719d-9fa0-4b6d-af47-e6b262bea8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:41.351019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca719d-9fa0-4b6d-af47-e6b262bea8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:41.375120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b7ca719d-9fa0-4b6d-af47-e6b262bea8d1 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:41.376636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b7ca719d-9fa0-4b6d-af47-e6b262bea8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:41.473335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 607/1202] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:13:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:41.504671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c1eeb-3ac5-4506-b2b3-ad67ba562881 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:41.654688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c1eeb-3ac5-4506-b2b3-ad67ba562881 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:41.708204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:41.709025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:41.709363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:41.709713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:41.723130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:41.725541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:41.756631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:41.757835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:41.759486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:41.759486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:41.759486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:41.759486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:41.759486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 596/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:41.767206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca719d-9fa0-4b6d-af47-e6b262bea8d1 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:41.924270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca719d-9fa0-4b6d-af47-e6b262bea8d1 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:41.962895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:41.969445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242381a-a824-464c-97cd-24a17c7c6a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:41.970123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242381a-a824-464c-97cd-24a17c7c6a06 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:41.970394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242381a-a824-464c-97cd-24a17c7c6a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:41.970674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242381a-a824-464c-97cd-24a17c7c6a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:41.971410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242381a-a824-464c-97cd-24a17c7c6a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:41.971718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242381a-a824-464c-97cd-24a17c7c6a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:41.973123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242381a-a824-464c-97cd-24a17c7c6a06 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:41.987389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 608/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:41.993634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:42.061241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:42.118664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274ba4c-17df-4a09-b4eb-fe015c076136 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:42.158567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d041cad-2859-4324-8b46-1e6860d32dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:42.159303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d041cad-2859-4324-8b46-1e6860d32dfd None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:42.159619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d041cad-2859-4324-8b46-1e6860d32dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:42.159988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d041cad-2859-4324-8b46-1e6860d32dfd None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:42.160832 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d041cad-2859-4324-8b46-1e6860d32dfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:42.162373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d041cad-2859-4324-8b46-1e6860d32dfd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:42.190170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 597/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Fri May 27 03:13:41 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:13:44.556454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6874510-fe00-44ac-9beb-99e343a5eb20 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:44.557011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6874510-fe00-44ac-9beb-99e343a5eb20 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:44.557011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6874510-fe00-44ac-9beb-99e343a5eb20 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:44.558651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 609/1206] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:44.563971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3f1f-cda6-4245-839d-847a91917d20 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:44.564402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3f1f-cda6-4245-839d-847a91917d20 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:44.564538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3f1f-cda6-4245-839d-847a91917d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:44.585452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-471c3f1f-cda6-4245-839d-847a91917d20 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:44.587058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-471c3f1f-cda6-4245-839d-847a91917d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:44.672449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 598/1207] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:13:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:44.698827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6874510-fe00-44ac-9beb-99e343a5eb20 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:44.830002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6874510-fe00-44ac-9beb-99e343a5eb20 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:44.865632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:44.866470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:44.866788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:44.867018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:44.878442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:44.887381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:44.888459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:44.889345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 610/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:13:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:44.894910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c3f1f-cda6-4245-839d-847a91917d20 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:44.949449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:45.017812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c3f1f-cda6-4245-839d-847a91917d20 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:45.056523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:45.057332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:45.057617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:45.057899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:45.058659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:45.058972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:45.060384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:45.069187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 599/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:13:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:45.075486 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:45.134949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:45.190812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc21d3-58d6-4e52-bc71-7a6269fdeff3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:45.232066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:45.232871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:45.233169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:45.233486 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:45.247290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:45.248871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:45.269314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:45.270497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:45.276205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:45.276205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:45.276205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:45.276205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:45.276205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 611/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:45.276562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:45.400281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:45.403226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4ddf6-3816-4d80-a71b-e16ac2c9de13 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:45.448549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:45.449695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:45.450273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:45.450716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:45.451649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:45.452268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:45.454252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:45.470777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 600/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:45.477700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:45.606733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091519-314a-4c2b-b77f-345d7538e27e None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:45.609277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:45.646370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:45.647310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:45.647581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:45.647942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:45.657059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:45.657206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:45.673582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:45.674931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:45.676328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:45.679554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 612/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:45.681105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:45.796957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac532b4-5d17-4c9e-acf3-cd6e43159a89 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:45.839890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:45.840577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:45.840890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:45.841165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:45.842003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:45.842319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:45.843785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:45.862067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 601/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Fri May 27 03:13:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:45.868497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:45.980149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db793104-41e1-4d49-be2c-9082b83e1553 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:46.023803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:46.024921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:46.025208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:46.025486 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:46.059056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:46.106059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:46.107615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:46.215904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 613/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:13:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:13:46.221767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:46.241769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:46.335260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a0b87-0197-4911-8b46-95cf4343f980 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:46.381950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:46.384138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:46.384447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:46.384730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:46.396038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:46.397512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:46.416017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:46.422777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:46.423736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:46.423736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:46.423736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:46.423736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:46.423736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 602/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:46.424073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:46.490180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:46.533910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de713-f2f7-4a1f-a84d-286aff1af177 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:46.568772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:46.569441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:46.569730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:46.570012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:46.571252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:46.571565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:46.573107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:46.590021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 614/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:46.591882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:46.663964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:46.697290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf61145-3e28-4a45-9e66-e0fa588c53db None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:46.732646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:46.733366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:46.733648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:46.733926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:46.743757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:46.744253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:13:46.761019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:46.762019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:46.764808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 603/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:46.768066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:46.889153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376eb17-9bc0-4776-b7d1-190bb230077f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:46.920956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:46.932630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bb282-3d23-4fb4-9719-e97997cfc263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:46.932866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bb282-3d23-4fb4-9719-e97997cfc263 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:46.932977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bb282-3d23-4fb4-9719-e97997cfc263 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:46.933233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bb282-3d23-4fb4-9719-e97997cfc263 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:46.934412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bb282-3d23-4fb4-9719-e97997cfc263 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:46.934832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bb282-3d23-4fb4-9719-e97997cfc263 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:46.936351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bb282-3d23-4fb4-9719-e97997cfc263 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:46.949926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 615/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:46.957264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:47.072862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d5879-a345-44ff-8375-81ef0b971e25 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:47.106943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:47.114094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05258c23-d511-4442-abbb-af24ffc60a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:47.114756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05258c23-d511-4442-abbb-af24ffc60a5a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:47.115042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05258c23-d511-4442-abbb-af24ffc60a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:47.115322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05258c23-d511-4442-abbb-af24ffc60a5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:47.116196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05258c23-d511-4442-abbb-af24ffc60a5a None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=8e6a5d72a7da4b07ac309bf308fc4745) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:47.116515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05258c23-d511-4442-abbb-af24ffc60a5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:47.117986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05258c23-d511-4442-abbb-af24ffc60a5a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:47.125632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 604/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:13:46 2022] GET /identity/v3/role_assignments?user.id=8e6a5d72a7da4b07ac309bf308fc4745&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:49.490154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5ac9-2a03-4093-839c-0060ee89eb01 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:49.490729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5ac9-2a03-4093-839c-0060ee89eb01 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:49.490729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5ac9-2a03-4093-839c-0060ee89eb01 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:49.492800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 616/1220] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:49.499938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fa7c3-2f7d-49f8-b7af-a95dfe5b61f6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:49.500485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fa7c3-2f7d-49f8-b7af-a95dfe5b61f6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:49.500485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fa7c3-2f7d-49f8-b7af-a95dfe5b61f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:49.519865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c72fa7c3-2f7d-49f8-b7af-a95dfe5b61f6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:49.521190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c72fa7c3-2f7d-49f8-b7af-a95dfe5b61f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:49.597961 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 605/1221] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:13:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:49.623687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a5ac9-2a03-4093-839c-0060ee89eb01 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:49.737976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a5ac9-2a03-4093-839c-0060ee89eb01 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:49.777396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:49.778115 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:49.778402 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:49.778680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:49.788707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:49.796687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:49.797855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:49.798712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 617/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:13:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:49.804938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fa7c3-2f7d-49f8-b7af-a95dfe5b61f6 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:49.908023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fa7c3-2f7d-49f8-b7af-a95dfe5b61f6 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:49.933088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:49.945887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:49.946595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:49.946880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:49.947220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:49.948043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] RBAC: Adding query filter params (name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:49.948319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:49.949807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:49.958777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 606/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:13:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:49.964899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:50.013511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:50.076445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55aedd-173d-4162-8dc9-b25464a56d71 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:50.119227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:50.120197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:50.120484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:50.120765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:50.135063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:50.137039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:50.155551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:50.156530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:50.157457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:50.157457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:50.157457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:50.157457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:50.157457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 618/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:50.162589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:50.273573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f8f8d-77ac-4f03-95d9-74c01e735512 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:50.311716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:50.312415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:50.312696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:50.312972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:50.313688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:50.313992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:50.315383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:50.331858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 607/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:50.332966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:50.428835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:50.436769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b255f4a-5c44-4456-8f02-422f9988cc35 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:50.474162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:50.474978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:50.475254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:50.475528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:50.484721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:50.486511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:50.501617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:50.503551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:50.505044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 619/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:50.509082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:50.562641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:50.633298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2680e-585f-4b96-a469-18366f42a961 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:50.673132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:50.673885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:50.674166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:50.674485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:50.675407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:50.675748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:50.677304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:50.704841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 608/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Fri May 27 03:13:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:50.706116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:50.820383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873e605-4897-46fe-9378-b7a31d4868b3 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:50.856821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:50.857623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:50.857919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:50.858184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:50.870185 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:50.871791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:50.892316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:50.893452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:50.894565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:50.894565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:50.894565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:50.894565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:50.895985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 620/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:50.902396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:51.019505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16300de-a86a-4216-8509-9f621de374e7 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:51.022046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:51.057921 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:51.058727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:51.059014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:51.059292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:51.060040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:51.060368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:51.061841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:51.074005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 609/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:51.080488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:51.095997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:51.194326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85cf73-cc90-40ec-adc0-0165f18af401 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:51.232379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:51.233246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:51.233531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:51.233811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:51.240620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:51.242042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:13:51.256181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:51.257286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:51.258301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 621/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:51.263802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:51.374122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8817ef-bf70-4621-8ad0-bdf8a26ef7bc None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:51.411055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbcba9-30f8-4731-81b2-b08efde2c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:51.411734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbcba9-30f8-4731-81b2-b08efde2c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:51.412042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbcba9-30f8-4731-81b2-b08efde2c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:51.412323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbcba9-30f8-4731-81b2-b08efde2c2a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:51.413113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbcba9-30f8-4731-81b2-b08efde2c2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:51.413483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbcba9-30f8-4731-81b2-b08efde2c2a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:51.415408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbcba9-30f8-4731-81b2-b08efde2c2a7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:51.428607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 610/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:13:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:51.434869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:51.507659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:51.544906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9af243-ee97-4a13-96d6-7776854bf9d0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:51.582049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f953f1f-d343-437c-89d1-40fb5014c2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:51.582989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f953f1f-d343-437c-89d1-40fb5014c2dd None admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:51.583337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f953f1f-d343-437c-89d1-40fb5014c2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:51.583616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f953f1f-d343-437c-89d1-40fb5014c2dd None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/8e6a5d72a7da4b07ac309bf308fc4745/roles/cf8ecb4f65d74cccbc07ab4e466a76ad` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:51.601471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f953f1f-d343-437c-89d1-40fb5014c2dd None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=8e6a5d72a7da4b07ac309bf308fc4745, role_id=cf8ecb4f65d74cccbc07ab4e466a76ad)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:51.602994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f953f1f-d343-437c-89d1-40fb5014c2dd None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:51.619145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 622/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:13:51 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/8e6a5d72a7da4b07ac309bf308fc4745/roles/cf8ecb4f65d74cccbc07ab4e466a76ad => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:13:53.903242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba49bc-475e-44cb-a438-2cc8f943c63f None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:53.903977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba49bc-475e-44cb-a438-2cc8f943c63f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:53.903977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba49bc-475e-44cb-a438-2cc8f943c63f None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:53.905235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 611/1233] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:53.911088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f129d2-9e5c-4597-a2fd-336b6762cef7 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:53.911596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f129d2-9e5c-4597-a2fd-336b6762cef7 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:53.911696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f129d2-9e5c-4597-a2fd-336b6762cef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:53.929957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c9f129d2-9e5c-4597-a2fd-336b6762cef7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:53.931181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c9f129d2-9e5c-4597-a2fd-336b6762cef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:54.011968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 623/1234] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:13:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:54.043808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba49bc-475e-44cb-a438-2cc8f943c63f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:54.080065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:54.170760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba49bc-475e-44cb-a438-2cc8f943c63f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:54.210050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:54.210971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.211256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.211529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.223379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:54.230927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:54.232031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:54.233446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 612/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Fri May 27 03:13:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:54.238457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f129d2-9e5c-4597-a2fd-336b6762cef7 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:54.285505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:54.344219 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f129d2-9e5c-4597-a2fd-336b6762cef7 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:54.381994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:54.383493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.384070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.384558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.385959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] RBAC: Adding query filter params (name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:54.386496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:54.389079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:54.402785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 624/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Fri May 27 03:13:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:54.408460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:54.506089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:54.525762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fe638-7cb4-47da-9638-dedabf4ec336 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:54.561194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:54.561982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.562277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.562553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.573780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:54.575232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:54.592528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:54.593775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:54.595032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:54.595032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:54.595032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:54.595032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:54.595032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 613/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:54.599566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:54.707916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ff9a4-1a52-4100-bb5c-06100a189174 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:54.717942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:54.744745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:54.745458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.745732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.746010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.746718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:54.747027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:54.748537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:54.761122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 625/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:54.766990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:54.878454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a0dd9-3726-431f-9f0f-9cefb78849b6 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:54.917805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:54.918453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.918453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.919641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.922265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:54.928177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:54.930261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:54.946702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:54.952985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:54.954482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 614/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:54.956497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:55.072018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337f000-da70-4762-8c49-bf2541b4b855 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:55.108822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:55.109541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:55.109854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:55.110168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:55.111093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:55.111467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:55.113058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:55.130443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 626/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Fri May 27 03:13:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:55.136611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:55.247476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda57dfe-3963-4fd8-b711-df068d6f4a7b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:55.287355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:55.288459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:55.288935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:55.289216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:55.357948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:55.364805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:55.366960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:55.479724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 615/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:13:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:13:55.487923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:55.520151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:55.603374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b76b1-a238-41f8-82a4-64e7fe05ded5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:55.641973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:55.642737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:55.643056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:55.643332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:55.654147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:55.655616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:55.674097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:55.675349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:55.676352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:55.676352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:55.676352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:55.676352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:55.676352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 627/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:55.681779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:55.802412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:55.814877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb7fe-bc28-4f0e-b612-7a022d49d6a3 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:55.853145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:55.853858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:55.854157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:55.854457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:55.855230 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:55.855543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:55.857021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:55.869519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 616/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:55.875321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:55.962174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:55.986937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d448b-47d1-4230-87e3-68597847ef4f None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:56.024822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:56.026011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:56.026282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:56.026585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:56.034130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:56.035737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:13:56.051570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:56.052530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:13:56.053447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 628/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:56.059841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:56.170866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe64d07-44c1-4992-ac1f-883fb24fd284 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:56.209632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8a6aa-576a-46de-a04c-b32d2f6ff9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:56.210277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8a6aa-576a-46de-a04c-b32d2f6ff9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:56.210631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8a6aa-576a-46de-a04c-b32d2f6ff9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:56.210927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8a6aa-576a-46de-a04c-b32d2f6ff9f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:56.211793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8a6aa-576a-46de-a04c-b32d2f6ff9f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:56.212142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8a6aa-576a-46de-a04c-b32d2f6ff9f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:56.213745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8a6aa-576a-46de-a04c-b32d2f6ff9f3 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:56.227743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 617/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:13:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:56.233552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:56.347072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584c389-7973-44dd-badd-55f62ca759ce None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:56.383838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf73f67-12db-4845-a08b-43fa3421114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:56.384561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf73f67-12db-4845-a08b-43fa3421114e None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:56.384854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf73f67-12db-4845-a08b-43fa3421114e None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:56.385169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf73f67-12db-4845-a08b-43fa3421114e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:56.386074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf73f67-12db-4845-a08b-43fa3421114e None admin] RBAC: Adding query filter params (role.id=cf8ecb4f65d74cccbc07ab4e466a76ad, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=8e6a5d72a7da4b07ac309bf308fc4745) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:56.386391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf73f67-12db-4845-a08b-43fa3421114e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:56.387986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf73f67-12db-4845-a08b-43fa3421114e None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:56.394636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 629/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:13:56 2022] GET /identity/v3/role_assignments?user.id=8e6a5d72a7da4b07ac309bf308fc4745&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=cf8ecb4f65d74cccbc07ab4e466a76ad => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:58.726400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b382a-f005-4911-abc5-f6134bc24813 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:58.727080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b382a-f005-4911-abc5-f6134bc24813 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:58.727080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b382a-f005-4911-abc5-f6134bc24813 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:58.728306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 618/1247] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:58.734573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9ef28-2b3c-4fc4-b8c9-1440d94ce1b0 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:58.735066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9ef28-2b3c-4fc4-b8c9-1440d94ce1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:58.735166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9ef28-2b3c-4fc4-b8c9-1440d94ce1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:58.741439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:58.756048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-47a9ef28-2b3c-4fc4-b8c9-1440d94ce1b0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:58.757799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-47a9ef28-2b3c-4fc4-b8c9-1440d94ce1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:58.840714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 630/1248] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:13:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:58.867551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b382a-f005-4911-abc5-f6134bc24813 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:58.887083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:58.986299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b382a-f005-4911-abc5-f6134bc24813 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:59.025542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:59.026498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:59.026809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:59.027090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:59.039746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:59.048236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:59.049530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:59.050432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 619/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:59.057097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9ef28-2b3c-4fc4-b8c9-1440d94ce1b0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:59.174072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9ef28-2b3c-4fc4-b8c9-1440d94ce1b0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:59.210730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:59.216197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:59.216871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:59.217157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:59.217438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:59.218197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:59.218510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:59.219908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:59.233819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 631/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:59.235106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:59.320280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:59.346170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658069c-27f9-4c5f-8b58-015e5a821a9e None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:59.383119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:59.384181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:59.384540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:59.384825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:59.395503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:59.397313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:59.415826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:59.421647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:59.422818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:59.422818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:59.422818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:59.422818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:59.422818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 620/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:59.423177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:59.532057 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e1fb1-f2e3-46b0-83b9-46c35c3721be None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:59.571836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:59.572615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:59.573024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:59.573337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:59.574337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:59.574710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:59.576455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:59.591797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 632/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:59.597387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:59.707846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22d5f2-233c-414a-ab79-69b66a9fb92f None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:59.746806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:59.760396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:59.761540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:59.761952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:59.762267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] PATH_INFO: `/v3/users/placement` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:59.773843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:59.775914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:59.794460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:59.795548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:13:59.797616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 621/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:13:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:59.801884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:59.864004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:59.914389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4f35f-a990-4663-b85d-5d957872c750 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:59.955365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:59.956068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:59.956359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:59.956655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:59.957597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:59.957910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:59.959366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:59.979448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 633/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Fri May 27 03:13:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:59.985847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:00.097035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc80ac7-2dd8-4991-b365-fff85d4de1af None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:00.133667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:00.134245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.134522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.134772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:00.190071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:00.205366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:00.207053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:00.312180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 622/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:13:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:14:00.318239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:00.427117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fdf3c-71d0-4ff8-99d7-6a29460f8a25 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:00.441547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:00.466159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:00.466956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.467239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.467512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:00.478686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:00.480312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:14:00.498157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:00.499489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:00.500680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:14:00.500680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:14:00.500680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:00.500680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:00.500680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 634/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:00.505603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:00.597927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:00.616304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705442ac-7c68-438c-bda3-5ee06d23730a None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:00.652578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:00.653325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.653607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.653878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:00.654608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:00.654918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:00.657587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:00.670111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 623/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:00.676188 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:00.801492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73416b09-ac8a-4dfa-87c6-946c6b7cb9d5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:00.838305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:00.839130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.839415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.839693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:00.847320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:00.848823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:14:00.863985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:00.865160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:14:00.866334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:00.868007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 635/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:14:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:00.871306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:00.976581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88247ec-74af-4ce0-be60-2f10013d0c07 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:01.011465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2431b3b-9611-40c2-8865-0637b0c07877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:01.012179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431b3b-9611-40c2-8865-0637b0c07877 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.012467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431b3b-9611-40c2-8865-0637b0c07877 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.012748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431b3b-9611-40c2-8865-0637b0c07877 None admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.013554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2431b3b-9611-40c2-8865-0637b0c07877 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:01.013869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2431b3b-9611-40c2-8865-0637b0c07877 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:01.015336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2431b3b-9611-40c2-8865-0637b0c07877 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:01.034059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 624/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:01.035205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:01.063400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:01.141727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef007b-7f4e-4f4f-ad69-913c90d8f4d6 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:01.175890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71801e6-c475-47a4-84cb-40dbfe98434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:01.176795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71801e6-c475-47a4-84cb-40dbfe98434f None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.177077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71801e6-c475-47a4-84cb-40dbfe98434f None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.177355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71801e6-c475-47a4-84cb-40dbfe98434f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.178194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71801e6-c475-47a4-84cb-40dbfe98434f None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=8e6a5d72a7da4b07ac309bf308fc4745) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:01.178511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71801e6-c475-47a4-84cb-40dbfe98434f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:01.179973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71801e6-c475-47a4-84cb-40dbfe98434f None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:01.185398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 636/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:14:01 2022] GET /identity/v3/role_assignments?user.id=8e6a5d72a7da4b07ac309bf308fc4745&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:03.358137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225eae-d392-43e3-bf7e-f677ecbe9e5b None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:03.358680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225eae-d392-43e3-bf7e-f677ecbe9e5b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:03.358680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225eae-d392-43e3-bf7e-f677ecbe9e5b None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:03.360100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 625/1261] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:03.365783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d9741-733c-4b20-ae98-1f8841839527 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:03.366218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d9741-733c-4b20-ae98-1f8841839527 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:03.366353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d9741-733c-4b20-ae98-1f8841839527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:03.383420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-249d9741-733c-4b20-ae98-1f8841839527 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:03.384623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-249d9741-733c-4b20-ae98-1f8841839527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:03.449460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 637/1262] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:14:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:14:03.471227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225eae-d392-43e3-bf7e-f677ecbe9e5b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:03.482269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:03.568698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225eae-d392-43e3-bf7e-f677ecbe9e5b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:03.601093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:03.601861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:03.602141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:03.602415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:03.612100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:03.620034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:03.621141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:14:03.622336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 626/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:14:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:03.627045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d9741-733c-4b20-ae98-1f8841839527 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:03.644763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:03.721474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d9741-733c-4b20-ae98-1f8841839527 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:03.759748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:03.760499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:03.760787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:03.761112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:03.761865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:03.762182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:03.763544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:03.772519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 638/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:14:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:03.779029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:03.846878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:03.882634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637fdd8-7594-41f7-a67a-dd6a0091c9ae None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:03.915524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:03.916350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:03.916635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:03.916910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:03.927379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:03.928857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:14:03.945474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:03.946520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:03.947810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:14:03.947810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:14:03.947810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:03.947810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:14:03.947810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 627/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:03.952359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:04.030888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:04.049778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300bdba-09dc-491e-abb6-7450f19fa79c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:04.081668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:04.082317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:04.082605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:04.082885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:04.083607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:04.084456 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:04.085830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:04.097204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 639/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:04.103203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:04.198899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d505478-24d4-48bc-a5a5-9b624490d197 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:04.217951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:04.232166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:04.232950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:04.233231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:04.233505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:04.240664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:04.242083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:14:04.255748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:04.257403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:14:04.258599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 628/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:14:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:04.265222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:04.360053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900f05b-9dd4-4d3a-adcf-e02865c7e9ad None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:04.392771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:04.393432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:04.393716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:04.393995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:04.394831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:04.395146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:04.397162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:04.405427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:04.414050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 640/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Fri May 27 03:14:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:04.419458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:04.515874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b6791-277b-4ef0-9a75-066ec3b3f2a2 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:04.548131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:04.548903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:04.549190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:04.549472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:04.559689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:04.561152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:14:04.578443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:04.579464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:04.580764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:14:04.580764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:14:04.580764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:04.580764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:14:04.580764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 629/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:04.585607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:04.680479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4075d-6891-41e6-a6d6-40c54709e7c5 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:04.713114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:04.713760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:04.714046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:04.714324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:04.715044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:04.715355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:04.716741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:04.728592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 641/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:04.734458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:04.769776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:04.836653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e243e3-186b-4fb4-aa43-d5eb32f98278 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:04.868934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:04.869713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:04.870001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:04.870282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:04.876760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:04.878197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:14:04.892616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:04.893509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:14:04.894447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 630/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:14:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:04.899596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:04.956268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:04.997706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ecc70-9ab6-49ff-b6c1-abfada5e698c None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:05.029900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba23c52-1e58-4949-af50-cf4f3737e383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:05.030490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba23c52-1e58-4949-af50-cf4f3737e383 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.030770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba23c52-1e58-4949-af50-cf4f3737e383 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.031045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba23c52-1e58-4949-af50-cf4f3737e383 None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.031842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba23c52-1e58-4949-af50-cf4f3737e383 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:05.032156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba23c52-1e58-4949-af50-cf4f3737e383 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:05.033585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba23c52-1e58-4949-af50-cf4f3737e383 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:05.045530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 642/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:14:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:05.051414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:05.138938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:05.147895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e390-78db-45f7-9407-ac550c5b71bd None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:05.180761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3283c-71d0-4eff-a6ec-d71307142265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:05.181629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3283c-71d0-4eff-a6ec-d71307142265 None admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.181910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3283c-71d0-4eff-a6ec-d71307142265 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.182185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3283c-71d0-4eff-a6ec-d71307142265 None admin] PATH_INFO: `/v3/projects/246f0e2eb19d40088d49d107007110e5/users/8e6a5d72a7da4b07ac309bf308fc4745/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.184123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3283c-71d0-4eff-a6ec-d71307142265 None admin] RBAC: Authorizing `identity:create_grant(project_id=246f0e2eb19d40088d49d107007110e5, user_id=8e6a5d72a7da4b07ac309bf308fc4745, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:05.185551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3283c-71d0-4eff-a6ec-d71307142265 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:05.198031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 631/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Fri May 27 03:14:05 2022] PUT /identity/v3/projects/246f0e2eb19d40088d49d107007110e5/users/8e6a5d72a7da4b07ac309bf308fc4745/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:07.326305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fe26e-7d70-4c90-98c3-bbd6a2cb3c3c None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:07.326819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fe26e-7d70-4c90-98c3-bbd6a2cb3c3c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:07.326819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fe26e-7d70-4c90-98c3-bbd6a2cb3c3c None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:07.328411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 643/1274] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:07.333489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e49fad-eb85-47c9-ae61-16b825abab92 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:07.333887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e49fad-eb85-47c9-ae61-16b825abab92 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:07.334045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e49fad-eb85-47c9-ae61-16b825abab92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:07.350713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-56e49fad-eb85-47c9-ae61-16b825abab92 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:07.351910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-56e49fad-eb85-47c9-ae61-16b825abab92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:07.415360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 632/1275] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:14:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:14:07.438983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fe26e-7d70-4c90-98c3-bbd6a2cb3c3c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:07.535166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fe26e-7d70-4c90-98c3-bbd6a2cb3c3c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:07.553785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:07.570162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:07.570947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:07.571229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:07.571504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:07.581565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:07.589518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:07.590636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:07.591688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 644/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Fri May 27 03:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:07.596771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e49fad-eb85-47c9-ae61-16b825abab92 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:07.674054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:07.692313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e49fad-eb85-47c9-ae61-16b825abab92 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:07.726096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:07.726902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:07.727275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:07.727624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:07.728572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:07.728932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:07.730566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:07.740518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 633/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Fri May 27 03:14:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:07.746832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:07.854317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d0d3-c996-4289-bd0e-a5ef8ed2b7b0 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:07.886443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:07.887228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:07.887512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:07.887789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:07.898529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:07.900005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:14:07.917132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:07.918112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:07.919105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:14:07.919105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:14:07.919105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:07.919105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:07.919105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 645/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:07.924079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.020300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a945463-b783-41af-a974-f0083e447db4 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.052761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.053410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.053696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.053974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.054700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:08.055012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.056415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:08.067674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 634/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:08.073648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.103999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:08.171652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02436fe4-1acf-4e27-a1ad-ef8cd4a29a92 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.204026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.204821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.205104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.205379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.213152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.214580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:14:08.228474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:08.229621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:08.230601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 646/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:14:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:08.235333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.239047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:08.331209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582cde9-1473-4078-ae9c-197c9b4609c5 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.363375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.364043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.364328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.364602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.365966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:08.366272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.367687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:08.384658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 635/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Fri May 27 03:14:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:08.390411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.478975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:08.488833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0217b2e-65f7-4271-ba1f-c00fbfa419ff None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.522544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.523180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.523470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.523751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.589474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.590928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:08.675174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:08.686208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 647/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Fri May 27 03:14:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:14:08.692015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.752959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:08.789745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14e15c-07e4-4e77-8d4e-f62679ef61d9 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.821864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.822664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.822940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.823211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.833036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.834498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:14:08.851664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:08.852743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:08.853936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:14:08.853936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:14:08.853936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:08.853936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:14:08.853936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 636/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:08.858581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.956682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383a157-de7b-4fee-bdfe-7855a61ce223 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.989489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.990751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.991034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.991313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.992073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:08.992385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.993748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:09.004672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 648/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Fri May 27 03:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:09.010189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:09.104708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef57a3-d129-4478-8e4a-8841b14d246b None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:09.117816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:09.138813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:09.139596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:09.139919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:09.140202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:09.147336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:09.148809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:14:09.162909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application with Lock( May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:09.163945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: ERROR keystone.server.flask.application May 27 03:14:09.165039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 637/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Fri May 27 03:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:09.169890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:09.218484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:09.267023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8e16-36c5-4e8e-abd0-da684cfc3773 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:09.299513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f1e3b-b3d4-4e82-98c9-7685ffeb534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:09.300192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f1e3b-b3d4-4e82-98c9-7685ffeb534b None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:09.300479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f1e3b-b3d4-4e82-98c9-7685ffeb534b None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:09.300754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f1e3b-b3d4-4e82-98c9-7685ffeb534b None admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:09.301539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f1e3b-b3d4-4e82-98c9-7685ffeb534b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:09.301853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f1e3b-b3d4-4e82-98c9-7685ffeb534b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:09.303304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f1e3b-b3d4-4e82-98c9-7685ffeb534b None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:09.314784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 649/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Fri May 27 03:14:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:09.321018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:09.417745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1f2b-1403-4b4b-88e7-f87980ffcaee None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:09.449156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447d51d-97e2-43f6-8617-ed735024761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:09.449792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447d51d-97e2-43f6-8617-ed735024761d None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:09.450078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447d51d-97e2-43f6-8617-ed735024761d None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:09.450355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447d51d-97e2-43f6-8617-ed735024761d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:09.451185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447d51d-97e2-43f6-8617-ed735024761d None admin] RBAC: Adding query filter params (role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f, scope.project.id=246f0e2eb19d40088d49d107007110e5, user.id=8e6a5d72a7da4b07ac309bf308fc4745) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:09.451497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447d51d-97e2-43f6-8617-ed735024761d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:09.453226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447d51d-97e2-43f6-8617-ed735024761d None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:09.459347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 638/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Fri May 27 03:14:09 2022] GET /identity/v3/role_assignments?user.id=8e6a5d72a7da4b07ac309bf308fc4745&scope.project.id=246f0e2eb19d40088d49d107007110e5&role.id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:11.626274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebffafb-5175-484b-9c1d-fedb2c5e6e12 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:11.626806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebffafb-5175-484b-9c1d-fedb2c5e6e12 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:11.626933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebffafb-5175-484b-9c1d-fedb2c5e6e12 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:11.628276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 650/1288] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:11.634052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e6585-3909-4c40-882c-6f7ec583e532 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:11.634451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e6585-3909-4c40-882c-6f7ec583e532 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:11.634617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e6585-3909-4c40-882c-6f7ec583e532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:11.650863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-798e6585-3909-4c40-882c-6f7ec583e532 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:11.652047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-798e6585-3909-4c40-882c-6f7ec583e532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:11.680286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:11.716462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 639/1289] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:14:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:14:11.741610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebffafb-5175-484b-9c1d-fedb2c5e6e12 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:11.845381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebffafb-5175-484b-9c1d-fedb2c5e6e12 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:11.856257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:11.879162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:11.880055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:11.880331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:11.880610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:11.881265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:11.882629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:11.894121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:11.895420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:11.896552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 651/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:14:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:11.901361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e6585-3909-4c40-882c-6f7ec583e532 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:11.996215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e6585-3909-4c40-882c-6f7ec583e532 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:12.027857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20393c15-1cf6-4b98-8bc8-09b81479d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:12.028513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20393c15-1cf6-4b98-8bc8-09b81479d972 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.028794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20393c15-1cf6-4b98-8bc8-09b81479d972 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.029071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20393c15-1cf6-4b98-8bc8-09b81479d972 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.029775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20393c15-1cf6-4b98-8bc8-09b81479d972 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:12.030082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20393c15-1cf6-4b98-8bc8-09b81479d972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:12.031440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20393c15-1cf6-4b98-8bc8-09b81479d972 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:12.036123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 640/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Fri May 27 03:14:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:12.042277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:12.138488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b15d-00fc-40ff-98e8-996f1608c027 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:12.170943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f7606-1df9-4474-ba38-1288d71397c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:12.171582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7606-1df9-4474-ba38-1288d71397c7 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.171897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7606-1df9-4474-ba38-1288d71397c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.172179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7606-1df9-4474-ba38-1288d71397c7 None admin] PATH_INFO: `/v3/services` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.172884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f7606-1df9-4474-ba38-1288d71397c7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:12.173196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f7606-1df9-4474-ba38-1288d71397c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:12.174569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f7606-1df9-4474-ba38-1288d71397c7 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:12.178838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 652/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:14.338851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1567a-d6fc-4cd3-8a95-47bbb053d032 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.339422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1567a-d6fc-4cd3-8a95-47bbb053d032 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.339422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1567a-d6fc-4cd3-8a95-47bbb053d032 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.340787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 641/1293] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:14.346047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c9338-3152-4886-b60b-22b92954d943 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.346434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c9338-3152-4886-b60b-22b92954d943 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.346607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c9338-3152-4886-b60b-22b92954d943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.363744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7e3c9338-3152-4886-b60b-22b92954d943 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:14.365425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7e3c9338-3152-4886-b60b-22b92954d943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:14.394120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.430227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 653/1294] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:14:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:14:14.453112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1567a-d6fc-4cd3-8a95-47bbb053d032 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.482498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.549704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1567a-d6fc-4cd3-8a95-47bbb053d032 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.583701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5259ed-5f94-42b6-8859-52438ca7e9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.584369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5259ed-5f94-42b6-8859-52438ca7e9b7 None admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.584652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5259ed-5f94-42b6-8859-52438ca7e9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.584925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5259ed-5f94-42b6-8859-52438ca7e9b7 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.585645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5259ed-5f94-42b6-8859-52438ca7e9b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.586997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5259ed-5f94-42b6-8859-52438ca7e9b7 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.603987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 642/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Fri May 27 03:14:14 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:16.795406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25bb01-a23c-4f50-a02a-b5ff42e81806 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:16.796162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25bb01-a23c-4f50-a02a-b5ff42e81806 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:16.796280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25bb01-a23c-4f50-a02a-b5ff42e81806 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:16.798204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 654/1296] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:16.804488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e38e-e920-432d-bb40-7904cf814099 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:16.804932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e38e-e920-432d-bb40-7904cf814099 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:16.805029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e38e-e920-432d-bb40-7904cf814099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:16.823455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-5a07e38e-e920-432d-bb40-7904cf814099 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:16.824732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5a07e38e-e920-432d-bb40-7904cf814099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:16.897407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 643/1297] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:14:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:14:16.920025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25bb01-a23c-4f50-a02a-b5ff42e81806 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.018984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25bb01-a23c-4f50-a02a-b5ff42e81806 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:17.051360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:17.052154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.052439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.052713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] PATH_INFO: `/v3/services/placement` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.053362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:17.054735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:17.066194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:17.067891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:17.069108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 655/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:14:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:17.073910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07e38e-e920-432d-bb40-7904cf814099 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.092476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:17.169624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07e38e-e920-432d-bb40-7904cf814099 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:17.202333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:17.202979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.203260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.203535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.204264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:17.204569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:17.205915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:17.211419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 644/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Fri May 27 03:14:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:17.217178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.248355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:17.314905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e664b1-fafd-4152-8cd1-dc3c8e9317ce None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:17.347589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad566-8888-43b8-8d8c-7356d8a2bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:17.348291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad566-8888-43b8-8d8c-7356d8a2bc05 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.348574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad566-8888-43b8-8d8c-7356d8a2bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.348857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad566-8888-43b8-8d8c-7356d8a2bc05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.349643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ad566-8888-43b8-8d8c-7356d8a2bc05 None admin] RBAC: Adding query filter params (interface=public, service_id=52d0488845d54983adb9c87396f178ec, region_id=RegionOne) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:17.349961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ad566-8888-43b8-8d8c-7356d8a2bc05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:17.351317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ad566-8888-43b8-8d8c-7356d8a2bc05 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:17.356202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 656/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Fri May 27 03:14:17 2022] GET /identity/v3/endpoints?service_id=52d0488845d54983adb9c87396f178ec&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:17.361756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.439999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:17.458410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9dec-290b-4f0a-84bf-e1605acf0901 None admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:17.490582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1f877-a96f-4fae-a5cd-672e16e51522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:17.491224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f877-a96f-4fae-a5cd-672e16e51522 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.491511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f877-a96f-4fae-a5cd-672e16e51522 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.491786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f877-a96f-4fae-a5cd-672e16e51522 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.492499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1f877-a96f-4fae-a5cd-672e16e51522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:17.493858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1f877-a96f-4fae-a5cd-672e16e51522 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:17.500458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 645/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Fri May 27 03:14:17 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:14:19.642259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de4201-9ed4-412b-9691-24736de7f92b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.642866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de4201-9ed4-412b-9691-24736de7f92b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.642866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de4201-9ed4-412b-9691-24736de7f92b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.644234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 657/1302] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:19.649808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7ca50-fde4-4b95-aa14-168ca792f0c3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.650231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7ca50-fde4-4b95-aa14-168ca792f0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.650416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7ca50-fde4-4b95-aa14-168ca792f0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.667225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-52e7ca50-fde4-4b95-aa14-168ca792f0c3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:19.668427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-52e7ca50-fde4-4b95-aa14-168ca792f0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:19.732605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 646/1303] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:14:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:14:19.755152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de4201-9ed4-412b-9691-24736de7f92b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.846524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:19.853959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de4201-9ed4-412b-9691-24736de7f92b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.885429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.886196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.886476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.886744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.887386 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.888762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:19.900799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:19.901946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application with Lock( May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: ERROR keystone.server.flask.application May 27 03:14:19.902965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 658/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:19.907866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7ca50-fde4-4b95-aa14-168ca792f0c3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.981429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:20.003076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7ca50-fde4-4b95-aa14-168ca792f0c3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.036408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de2a9c-b8e0-4002-ba6f-59e086133764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.037101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de2a9c-b8e0-4002-ba6f-59e086133764 None admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.037387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de2a9c-b8e0-4002-ba6f-59e086133764 None admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.037666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de2a9c-b8e0-4002-ba6f-59e086133764 None admin] PATH_INFO: `/v3/services` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.038377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de2a9c-b8e0-4002-ba6f-59e086133764 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:20.038688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de2a9c-b8e0-4002-ba6f-59e086133764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.040078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de2a9c-b8e0-4002-ba6f-59e086133764 None admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.045094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 647/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:20.050750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.146708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae99ed-bfa2-4620-966c-1ee61108e9d2 None admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.179385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67374b4-8a50-4c4d-85ce-eebc82ae79c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.179992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67374b4-8a50-4c4d-85ce-eebc82ae79c6 None admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.180274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67374b4-8a50-4c4d-85ce-eebc82ae79c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.180550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67374b4-8a50-4c4d-85ce-eebc82ae79c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.181278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67374b4-8a50-4c4d-85ce-eebc82ae79c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.182622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67374b4-8a50-4c4d-85ce-eebc82ae79c6 None admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.199753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 659/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Fri May 27 03:14:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:30.042291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=30 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:30.195733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.174666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9fcab-5539-4e9e-b67a-7574f86e73f3 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.175879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9fcab-5539-4e9e-b67a-7574f86e73f3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.175879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9fcab-5539-4e9e-b67a-7574f86e73f3 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.181130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 648/1307] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:23.182498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd4e6-42ed-4ea1-a3eb-6db0d0f65cca None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.182946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd4e6-42ed-4ea1-a3eb-6db0d0f65cca None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.183149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd4e6-42ed-4ea1-a3eb-6db0d0f65cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.269385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-170dd4e6-42ed-4ea1-a3eb-6db0d0f65cca None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:23.270609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-170dd4e6-42ed-4ea1-a3eb-6db0d0f65cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:23.501787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 660/1308] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:23.570118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299232-3f07-4413-a101-0c9349cd9615 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.570596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299232-3f07-4413-a101-0c9349cd9615 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.571099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299232-3f07-4413-a101-0c9349cd9615 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.573322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 649/1309] 10.222.0.62 () {54 vars in 964 bytes} [Fri May 27 03:15:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:23.579900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4e33-de6f-499a-a4ad-120b207eb780 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.580497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4e33-de6f-499a-a4ad-120b207eb780 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.580923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4e33-de6f-499a-a4ad-120b207eb780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.754094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-29bc4e33-de6f-499a-a4ad-120b207eb780 None None] MFA Rules not processed for user `a364b6179abb4ad0883898090acef827`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:23.757330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-29bc4e33-de6f-499a-a4ad-120b207eb780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:23.803055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.895257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 661/1310] 10.222.0.62 () {58 vars in 1061 bytes} [Fri May 27 03:15:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:23.895873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95299232-3f07-4413-a101-0c9349cd9615 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:23.897692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-95299232-3f07-4413-a101-0c9349cd9615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:24.056086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95299232-3f07-4413-a101-0c9349cd9615 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.136208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033f884-3cba-44d3-bf9b-40042621b3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.136865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f884-3cba-44d3-bf9b-40042621b3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.137148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f884-3cba-44d3-bf9b-40042621b3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.137425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f884-3cba-44d3-bf9b-40042621b3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.139067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0033f884-3cba-44d3-bf9b-40042621b3d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:24.199387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.259858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033f884-3cba-44d3-bf9b-40042621b3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.261372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033f884-3cba-44d3-bf9b-40042621b3d4 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.282429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 650/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:26.868208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ddc8-7497-42ee-8754-44b048b08061 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.868782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ddc8-7497-42ee-8754-44b048b08061 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.868782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558ddc8-7497-42ee-8754-44b048b08061 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.870321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 662/1312] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:26.875320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d33f7-acd1-409e-94e9-a80d85e0c1ee None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.875786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d33f7-acd1-409e-94e9-a80d85e0c1ee None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.875909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d33f7-acd1-409e-94e9-a80d85e0c1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.952776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-bb8d33f7-acd1-409e-94e9-a80d85e0c1ee None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:26.953961 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-bb8d33f7-acd1-409e-94e9-a80d85e0c1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:27.049046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 651/1313] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:27.086944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558ddc8-7497-42ee-8754-44b048b08061 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.088503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e558ddc8-7497-42ee-8754-44b048b08061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:27.189194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558ddc8-7497-42ee-8754-44b048b08061 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.264116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.266693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90722b7-8480-4c64-bba2-a9789071eb73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.267775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90722b7-8480-4c64-bba2-a9789071eb73 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.268097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90722b7-8480-4c64-bba2-a9789071eb73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.268537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90722b7-8480-4c64-bba2-a9789071eb73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.270559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c90722b7-8480-4c64-bba2-a9789071eb73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:27.380973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90722b7-8480-4c64-bba2-a9789071eb73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.382420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90722b7-8480-4c64-bba2-a9789071eb73 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.400870 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 663/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:29.953052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421e659-47b5-44ac-a287-645d19e5c97f None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.953535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421e659-47b5-44ac-a287-645d19e5c97f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.953674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421e659-47b5-44ac-a287-645d19e5c97f None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.954966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 652/1315] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:29.961012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead8500-392c-409a-8942-e4ee3724e166 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.961451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead8500-392c-409a-8942-e4ee3724e166 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.961585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead8500-392c-409a-8942-e4ee3724e166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.041778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-aead8500-392c-409a-8942-e4ee3724e166 None None] MFA Rules not processed for user `c3a7322e88e7474ab0bccfd41305fae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:30.042981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-aead8500-392c-409a-8942-e4ee3724e166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:30.095428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.139995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 664/1316] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:30.179119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421e659-47b5-44ac-a287-645d19e5c97f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.180693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6421e659-47b5-44ac-a287-645d19e5c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:30.190428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.283813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421e659-47b5-44ac-a287-645d19e5c97f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:30.356448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ff0c9-bbc6-4349-9205-5affada46553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:30.357089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff0c9-bbc6-4349-9205-5affada46553 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.357371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff0c9-bbc6-4349-9205-5affada46553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.357645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff0c9-bbc6-4349-9205-5affada46553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.359256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-169ff0c9-bbc6-4349-9205-5affada46553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:30.419249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.465939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ff0c9-bbc6-4349-9205-5affada46553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:30.467300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ff0c9-bbc6-4349-9205-5affada46553 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.485101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 653/1317] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:33.223009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361904-0fb8-4fff-90f3-7d3e426e7af2 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:33.223514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361904-0fb8-4fff-90f3-7d3e426e7af2 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:33.223514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361904-0fb8-4fff-90f3-7d3e426e7af2 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:33.224951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 665/1318] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:33.230180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495b513-607d-4e65-a55d-8d89ac98f829 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:33.230598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495b513-607d-4e65-a55d-8d89ac98f829 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:33.230744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495b513-607d-4e65-a55d-8d89ac98f829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:33.293749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-7495b513-607d-4e65-a55d-8d89ac98f829 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:33.294938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7495b513-607d-4e65-a55d-8d89ac98f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:33.389130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 654/1319] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:33.427738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7361904-0fb8-4fff-90f3-7d3e426e7af2 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:33.429275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a7361904-0fb8-4fff-90f3-7d3e426e7af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:33.529543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7361904-0fb8-4fff-90f3-7d3e426e7af2 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:33.599494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b7084-edb8-4800-a862-bad8006ec185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:33.600145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b7084-edb8-4800-a862-bad8006ec185 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:33.600427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b7084-edb8-4800-a862-bad8006ec185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:33.600699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b7084-edb8-4800-a862-bad8006ec185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:33.602286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b69b7084-edb8-4800-a862-bad8006ec185 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:33.609367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:33.708407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b7084-edb8-4800-a862-bad8006ec185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:33.709820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b7084-edb8-4800-a862-bad8006ec185 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:33.726919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 666/1320] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:36.558996 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1287-c9e9-4928-a91c-853c8ec28a92 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.559598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1287-c9e9-4928-a91c-853c8ec28a92 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.559598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1287-c9e9-4928-a91c-853c8ec28a92 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.560960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 655/1321] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:36.566772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819de79-8159-4e22-809d-70dcba7e8232 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.567195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819de79-8159-4e22-809d-70dcba7e8232 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.567330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819de79-8159-4e22-809d-70dcba7e8232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.630520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4819de79-8159-4e22-809d-70dcba7e8232 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:36.631699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4819de79-8159-4e22-809d-70dcba7e8232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:36.688588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.728156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 667/1322] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:36.763744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b1287-c9e9-4928-a91c-853c8ec28a92 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.765283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dd7b1287-c9e9-4928-a91c-853c8ec28a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:36.784721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.867866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b1287-c9e9-4928-a91c-853c8ec28a92 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.941378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e813-0309-40f7-8edb-293066d5829b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.942483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e813-0309-40f7-8edb-293066d5829b service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.942764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e813-0309-40f7-8edb-293066d5829b service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.943038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e813-0309-40f7-8edb-293066d5829b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.944739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1be3e813-0309-40f7-8edb-293066d5829b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:37.016332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.055396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3e813-0309-40f7-8edb-293066d5829b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.056875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3e813-0309-40f7-8edb-293066d5829b service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.076041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 656/1323] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:39.335781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20d5f0-d20d-43f3-8914-40acdb577a26 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.336353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20d5f0-d20d-43f3-8914-40acdb577a26 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.336458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20d5f0-d20d-43f3-8914-40acdb577a26 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.337814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 668/1324] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:39.343133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddd38c-d482-4868-8c3b-4b12ed06984d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.343666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddd38c-d482-4868-8c3b-4b12ed06984d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.343666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddd38c-d482-4868-8c3b-4b12ed06984d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.406624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-09ddd38c-d482-4868-8c3b-4b12ed06984d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:39.407802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-09ddd38c-d482-4868-8c3b-4b12ed06984d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:39.501806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 657/1325] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:39.537139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20d5f0-d20d-43f3-8914-40acdb577a26 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:39.538704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0f20d5f0-d20d-43f3-8914-40acdb577a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:39.640005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20d5f0-d20d-43f3-8914-40acdb577a26 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:39.711593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3891855-1489-4990-bb01-1c2b7ee354b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:39.712275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891855-1489-4990-bb01-1c2b7ee354b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.712557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891855-1489-4990-bb01-1c2b7ee354b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.712831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891855-1489-4990-bb01-1c2b7ee354b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.714444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a3891855-1489-4990-bb01-1c2b7ee354b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:39.730610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:39.822536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3891855-1489-4990-bb01-1c2b7ee354b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:39.823906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3891855-1489-4990-bb01-1c2b7ee354b3 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:39.842562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 669/1326] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:42.083391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04502ae-e204-4475-9bbd-7e5cbb84b962 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.083959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04502ae-e204-4475-9bbd-7e5cbb84b962 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.083959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04502ae-e204-4475-9bbd-7e5cbb84b962 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.085358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 658/1327] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:42.090719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f09fd7-bdfd-4aba-b1e2-0fe1a0f416f5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.091156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f09fd7-bdfd-4aba-b1e2-0fe1a0f416f5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.091306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f09fd7-bdfd-4aba-b1e2-0fe1a0f416f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.156752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d3f09fd7-bdfd-4aba-b1e2-0fe1a0f416f5 None None] MFA Rules not processed for user `c3a7322e88e7474ab0bccfd41305fae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:42.158028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d3f09fd7-bdfd-4aba-b1e2-0fe1a0f416f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:42.223272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.256221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 670/1328] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:42.292911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04502ae-e204-4475-9bbd-7e5cbb84b962 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.294478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d04502ae-e204-4475-9bbd-7e5cbb84b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:42.338026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.415662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04502ae-e204-4475-9bbd-7e5cbb84b962 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.490611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b70dfe-b868-4c22-b191-ce93d7acf042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.491183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b70dfe-b868-4c22-b191-ce93d7acf042 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.491467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b70dfe-b868-4c22-b191-ce93d7acf042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.491744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b70dfe-b868-4c22-b191-ce93d7acf042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.493410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-44b70dfe-b868-4c22-b191-ce93d7acf042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:42.570091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.602067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b70dfe-b868-4c22-b191-ce93d7acf042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.603506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b70dfe-b868-4c22-b191-ce93d7acf042 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.625811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 659/1329] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:45.045120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821e497-858d-4dba-b674-0b797bddd605 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:45.045618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821e497-858d-4dba-b674-0b797bddd605 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:45.045734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821e497-858d-4dba-b674-0b797bddd605 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:45.047095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 671/1330] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:45.053031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e897-6bc0-42db-8f6a-d4a1a2159b72 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:45.053493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e897-6bc0-42db-8f6a-d4a1a2159b72 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:45.053710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e897-6bc0-42db-8f6a-d4a1a2159b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:45.117032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9cb0e897-6bc0-42db-8f6a-d4a1a2159b72 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:45.118240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9cb0e897-6bc0-42db-8f6a-d4a1a2159b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:45.213370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 660/1331] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:45.248509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821e497-858d-4dba-b674-0b797bddd605 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:45.250114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4821e497-858d-4dba-b674-0b797bddd605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:45.361880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821e497-858d-4dba-b674-0b797bddd605 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:45.438737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e8815-0b79-4d35-a669-50d7e7c13592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:45.439381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8815-0b79-4d35-a669-50d7e7c13592 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:45.439662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8815-0b79-4d35-a669-50d7e7c13592 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:45.439978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8815-0b79-4d35-a669-50d7e7c13592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:45.441629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c83e8815-0b79-4d35-a669-50d7e7c13592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:45.469567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:45.568804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8815-0b79-4d35-a669-50d7e7c13592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:45.570357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8815-0b79-4d35-a669-50d7e7c13592 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:45.592067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 672/1332] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:47.835192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6453c4-1c72-43d1-9704-d9ae6b8fd653 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.835700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6453c4-1c72-43d1-9704-d9ae6b8fd653 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.835806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6453c4-1c72-43d1-9704-d9ae6b8fd653 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.837171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 661/1333] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:47.842236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad9d56-7aa5-4026-a0e0-14ec12d33346 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.842682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad9d56-7aa5-4026-a0e0-14ec12d33346 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.842808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad9d56-7aa5-4026-a0e0-14ec12d33346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.906651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-1cad9d56-7aa5-4026-a0e0-14ec12d33346 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:47.907862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1cad9d56-7aa5-4026-a0e0-14ec12d33346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:47.984253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.006983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 673/1334] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:48.045744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6453c4-1c72-43d1-9704-d9ae6b8fd653 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.047267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7f6453c4-1c72-43d1-9704-d9ae6b8fd653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:48.087846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.150383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6453c4-1c72-43d1-9704-d9ae6b8fd653 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.220876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a638d-cc5f-4baf-a2a5-d254c602d0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.221510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a638d-cc5f-4baf-a2a5-d254c602d0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.221792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a638d-cc5f-4baf-a2a5-d254c602d0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.222067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a638d-cc5f-4baf-a2a5-d254c602d0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.224303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-563a638d-cc5f-4baf-a2a5-d254c602d0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:48.305864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.331515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a638d-cc5f-4baf-a2a5-d254c602d0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.333516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a638d-cc5f-4baf-a2a5-d254c602d0ba service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.351517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 662/1335] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:50.837998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e5c74-5c9b-48e1-8e8f-e48528f3008a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.838544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e5c74-5c9b-48e1-8e8f-e48528f3008a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.838544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e5c74-5c9b-48e1-8e8f-e48528f3008a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.839957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 674/1336] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:50.845051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ecd98-400e-4fca-a72c-217979ee21b7 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.845437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ecd98-400e-4fca-a72c-217979ee21b7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.845610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ecd98-400e-4fca-a72c-217979ee21b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.908715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-276ecd98-400e-4fca-a72c-217979ee21b7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:50.909880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-276ecd98-400e-4fca-a72c-217979ee21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:51.003769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 663/1337] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:51.039176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e5c74-5c9b-48e1-8e8f-e48528f3008a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:51.040725 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5b9e5c74-5c9b-48e1-8e8f-e48528f3008a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:51.140175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e5c74-5c9b-48e1-8e8f-e48528f3008a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:51.210472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f21d5-c1c7-4edd-8953-27e9bc71201a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:51.211103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f21d5-c1c7-4edd-8953-27e9bc71201a service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:51.211383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f21d5-c1c7-4edd-8953-27e9bc71201a service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:51.211659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f21d5-c1c7-4edd-8953-27e9bc71201a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:51.213276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-007f21d5-c1c7-4edd-8953-27e9bc71201a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:51.247172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:51.318148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f21d5-c1c7-4edd-8953-27e9bc71201a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:51.319568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f21d5-c1c7-4edd-8953-27e9bc71201a service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:51.337730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 675/1338] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:55.445552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bccfa-4845-4618-8e0c-38c2ba323f7e None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:55.446086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bccfa-4845-4618-8e0c-38c2ba323f7e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:55.446086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bccfa-4845-4618-8e0c-38c2ba323f7e None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:55.447383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 664/1339] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:55.452892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4d25-e541-4bc8-8ae4-52a497694359 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:55.453315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4d25-e541-4bc8-8ae4-52a497694359 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:55.453593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4d25-e541-4bc8-8ae4-52a497694359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:55.522925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-29dd4d25-e541-4bc8-8ae4-52a497694359 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:55.524050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-29dd4d25-e541-4bc8-8ae4-52a497694359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:55.616022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:55.629156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 676/1340] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:55.668200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bccfa-4845-4618-8e0c-38c2ba323f7e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:55.670342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a54bccfa-4845-4618-8e0c-38c2ba323f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:55.720292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:55.770419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bccfa-4845-4618-8e0c-38c2ba323f7e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:55.774114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16e06d-f73a-49a7-89f1-a5d287d15b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:55.774718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16e06d-f73a-49a7-89f1-a5d287d15b2c service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:55.775005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16e06d-f73a-49a7-89f1-a5d287d15b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:55.775286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16e06d-f73a-49a7-89f1-a5d287d15b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:55.784826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16e06d-f73a-49a7-89f1-a5d287d15b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:55.787708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16e06d-f73a-49a7-89f1-a5d287d15b2c service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:55.798695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 665/1341] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:58.676801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5af0a-18f0-4ac1-923f-544785366c49 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.677397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5af0a-18f0-4ac1-923f-544785366c49 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.677397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5af0a-18f0-4ac1-923f-544785366c49 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.678753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 677/1342] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:58.684102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa489d97-7917-4076-962c-aaf846cdb5d2 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.684533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa489d97-7917-4076-962c-aaf846cdb5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.684789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa489d97-7917-4076-962c-aaf846cdb5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.712581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fa489d97-7917-4076-962c-aaf846cdb5d2 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:58.713758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fa489d97-7917-4076-962c-aaf846cdb5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:58.803586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 666/1343] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:15:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:58.840318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5af0a-18f0-4ac1-923f-544785366c49 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:58.851490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5af0a-18f0-4ac1-923f-544785366c49 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:58.855022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c0169-9c37-4149-af20-ea4f19014fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:58.855658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c0169-9c37-4149-af20-ea4f19014fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.856059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c0169-9c37-4149-af20-ea4f19014fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.856345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c0169-9c37-4149-af20-ea4f19014fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.865285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c0169-9c37-4149-af20-ea4f19014fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:58.866744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c0169-9c37-4149-af20-ea4f19014fc1 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:58.876116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 678/1344] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:15:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:02.947261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851d158-12ba-4fb4-bfb5-7cd8c3350034 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.947690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851d158-12ba-4fb4-bfb5-7cd8c3350034 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.947777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851d158-12ba-4fb4-bfb5-7cd8c3350034 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.949210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 667/1345] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:02.954537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9927d-6f5f-401c-9a1a-e9091cb03e3d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.954997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9927d-6f5f-401c-9a1a-e9091cb03e3d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.955360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9927d-6f5f-401c-9a1a-e9091cb03e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.974987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7bd9927d-6f5f-401c-9a1a-e9091cb03e3d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:02.976300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7bd9927d-6f5f-401c-9a1a-e9091cb03e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:02.984064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 679/1346] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:03.021000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851d158-12ba-4fb4-bfb5-7cd8c3350034 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.031711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851d158-12ba-4fb4-bfb5-7cd8c3350034 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.035106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ac7c2-bdc4-4b92-96c1-f27853eae121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.035732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ac7c2-bdc4-4b92-96c1-f27853eae121 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.036043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ac7c2-bdc4-4b92-96c1-f27853eae121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.036836 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ac7c2-bdc4-4b92-96c1-f27853eae121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.045532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ac7c2-bdc4-4b92-96c1-f27853eae121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.046959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ac7c2-bdc4-4b92-96c1-f27853eae121 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.056923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 668/1347] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:08.337728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a173e32-1eec-4eba-b2ca-f016387ca9ee None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:08.338234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a173e32-1eec-4eba-b2ca-f016387ca9ee None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:08.338316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a173e32-1eec-4eba-b2ca-f016387ca9ee None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:08.340549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 680/1348] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:08.345341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896db96-99aa-4241-b10f-9d7de9385392 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:08.345739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896db96-99aa-4241-b10f-9d7de9385392 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:08.345894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896db96-99aa-4241-b10f-9d7de9385392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:08.420774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-6896db96-99aa-4241-b10f-9d7de9385392 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:08.422380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6896db96-99aa-4241-b10f-9d7de9385392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:08.451365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:08.502407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 669/1349] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:16:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:12.880328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b0c79-46fe-4b62-ad16-aad66c76cfd8 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.881502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b0c79-46fe-4b62-ad16-aad66c76cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.881502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b0c79-46fe-4b62-ad16-aad66c76cfd8 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.883136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 681/1350] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:12.888605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43ca7-be52-4bd1-8917-ad7a0ff962b0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.889149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43ca7-be52-4bd1-8917-ad7a0ff962b0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.889149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43ca7-be52-4bd1-8917-ad7a0ff962b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.908558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c5d43ca7-be52-4bd1-8917-ad7a0ff962b0 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:12.909788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c5d43ca7-be52-4bd1-8917-ad7a0ff962b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:12.928661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 670/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:16:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:12.968317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=13 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:14.324553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e4a0-1147-4a31-a514-8120703c4361 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.325175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e4a0-1147-4a31-a514-8120703c4361 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.325175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e4a0-1147-4a31-a514-8120703c4361 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.326635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 682/1352] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:14.331982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8719-39ef-4eab-976d-c311c3444907 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.332382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8719-39ef-4eab-976d-c311c3444907 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.332673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8719-39ef-4eab-976d-c311c3444907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.351951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-200b8719-39ef-4eab-976d-c311c3444907 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:14.353205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-200b8719-39ef-4eab-976d-c311c3444907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:14.361639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 671/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:16:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:16:18.308688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f3e4-65bc-4cf0-8612-9872670f913e None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.309286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f3e4-65bc-4cf0-8612-9872670f913e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.309286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f3e4-65bc-4cf0-8612-9872670f913e None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.310936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 683/1354] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:18.316120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e037e3d-57e0-4a0b-9e0f-5935e336f292 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.316522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e037e3d-57e0-4a0b-9e0f-5935e336f292 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.316664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e037e3d-57e0-4a0b-9e0f-5935e336f292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.334949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0e037e3d-57e0-4a0b-9e0f-5935e336f292 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:18.336143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0e037e3d-57e0-4a0b-9e0f-5935e336f292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:18.344068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 672/1355] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:18.403206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381c9d1-0f01-4d5b-bab3-e0f5a3927ac5 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.403875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381c9d1-0f01-4d5b-bab3-e0f5a3927ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.404383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381c9d1-0f01-4d5b-bab3-e0f5a3927ac5 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.406125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 684/1356] 10.222.0.62 () {54 vars in 961 bytes} [Fri May 27 03:16:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:18.417126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a59046-fddc-47c9-b0cc-736a206ecb03 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.417689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a59046-fddc-47c9-b0cc-736a206ecb03 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.418567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a59046-fddc-47c9-b0cc-736a206ecb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.470562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e1a59046-fddc-47c9-b0cc-736a206ecb03 None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:18.471929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e1a59046-fddc-47c9-b0cc-736a206ecb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:18.569621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 673/1357] 10.222.0.62 () {58 vars in 1058 bytes} [Fri May 27 03:16:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:18.577673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381c9d1-0f01-4d5b-bab3-e0f5a3927ac5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.593807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381c9d1-0f01-4d5b-bab3-e0f5a3927ac5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.599771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ce27d-e9b2-472f-b30f-c97f60f13d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.600606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce27d-e9b2-472f-b30f-c97f60f13d25 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.601018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce27d-e9b2-472f-b30f-c97f60f13d25 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.601397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce27d-e9b2-472f-b30f-c97f60f13d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.610310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ce27d-e9b2-472f-b30f-c97f60f13d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.612087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ce27d-e9b2-472f-b30f-c97f60f13d25 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.621799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 685/1358] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:16:21.036782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf1184-dc6b-4e68-baee-01cdcdc89527 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.037814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf1184-dc6b-4e68-baee-01cdcdc89527 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.037814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf1184-dc6b-4e68-baee-01cdcdc89527 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.039292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 674/1359] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:21.044469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb430797-63c0-4ded-b743-09da91a1dc11 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.044857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb430797-63c0-4ded-b743-09da91a1dc11 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.045025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb430797-63c0-4ded-b743-09da91a1dc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.063556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fb430797-63c0-4ded-b743-09da91a1dc11 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:21.064790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fb430797-63c0-4ded-b743-09da91a1dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:21.072796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 686/1360] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:21.117665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf1184-dc6b-4e68-baee-01cdcdc89527 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:21.132339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf1184-dc6b-4e68-baee-01cdcdc89527 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:21.137819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c28af-5213-45fa-b838-54537205fb4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:21.138450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c28af-5213-45fa-b838-54537205fb4f service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.138732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c28af-5213-45fa-b838-54537205fb4f service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.139023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c28af-5213-45fa-b838-54537205fb4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.148266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c28af-5213-45fa-b838-54537205fb4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:21.149839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c28af-5213-45fa-b838-54537205fb4f service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:21.160091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 675/1361] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:23.695558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57d4a-7f9c-47f7-8687-d6a9b62a8fa3 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.696130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57d4a-7f9c-47f7-8687-d6a9b62a8fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.696130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57d4a-7f9c-47f7-8687-d6a9b62a8fa3 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.697537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 687/1362] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:23.702722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca40be-1161-470c-9c28-15e4b2498e84 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.703167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca40be-1161-470c-9c28-15e4b2498e84 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.703289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca40be-1161-470c-9c28-15e4b2498e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.727680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-42ca40be-1161-470c-9c28-15e4b2498e84 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:23.728906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-42ca40be-1161-470c-9c28-15e4b2498e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:23.738133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 676/1363] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:23.785604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332271d-a845-4ddd-a32e-08e9ad20a7c1 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.786007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332271d-a845-4ddd-a32e-08e9ad20a7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.786289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332271d-a845-4ddd-a32e-08e9ad20a7c1 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.787672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 688/1364] 10.222.0.62 () {54 vars in 961 bytes} [Fri May 27 03:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:23.793207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a50f-3da3-429e-a45e-9d5aeabcfd40 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.793684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a50f-3da3-429e-a45e-9d5aeabcfd40 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.794111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a50f-3da3-429e-a45e-9d5aeabcfd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.826682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8903a50f-3da3-429e-a45e-9d5aeabcfd40 None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:23.828180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8903a50f-3da3-429e-a45e-9d5aeabcfd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:23.841892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 677/1365] 10.222.0.62 () {58 vars in 1058 bytes} [Fri May 27 03:16:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:23.849865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332271d-a845-4ddd-a32e-08e9ad20a7c1 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:23.864559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332271d-a845-4ddd-a32e-08e9ad20a7c1 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.869862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58225baf-570d-43db-a24b-5d546d10b808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.870630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58225baf-570d-43db-a24b-5d546d10b808 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.870918 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58225baf-570d-43db-a24b-5d546d10b808 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.871201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58225baf-570d-43db-a24b-5d546d10b808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.879197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58225baf-570d-43db-a24b-5d546d10b808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.881437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58225baf-570d-43db-a24b-5d546d10b808 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.893332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 689/1366] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:26.181630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b5e41-f24e-42f4-9af5-49d6428457b8 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.182537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b5e41-f24e-42f4-9af5-49d6428457b8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.182820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b5e41-f24e-42f4-9af5-49d6428457b8 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.184300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 678/1367] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:26.189741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f372a-35db-493a-8979-a7a7aad66d71 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.190157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f372a-35db-493a-8979-a7a7aad66d71 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.190241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f372a-35db-493a-8979-a7a7aad66d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.208367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fe2f372a-35db-493a-8979-a7a7aad66d71 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:26.209621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fe2f372a-35db-493a-8979-a7a7aad66d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:26.219250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 690/1368] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:26.264067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b5e41-f24e-42f4-9af5-49d6428457b8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:26.278802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b5e41-f24e-42f4-9af5-49d6428457b8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:26.284386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ff0cd-7917-4835-907a-b49da0bdbaea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:26.285122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ff0cd-7917-4835-907a-b49da0bdbaea service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.285471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ff0cd-7917-4835-907a-b49da0bdbaea service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.285818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ff0cd-7917-4835-907a-b49da0bdbaea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.295336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ff0cd-7917-4835-907a-b49da0bdbaea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:26.297105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ff0cd-7917-4835-907a-b49da0bdbaea service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:26.308858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 679/1369] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:26.498218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf4e6a-e68e-47f3-b608-f68f52ca6e3b None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.498796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf4e6a-e68e-47f3-b608-f68f52ca6e3b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.499085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf4e6a-e68e-47f3-b608-f68f52ca6e3b None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.501068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 691/1370] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:26.518683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6bb2-1664-4c79-bf4d-74c3c0cc6728 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.519380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6bb2-1664-4c79-bf4d-74c3c0cc6728 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.519668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6bb2-1664-4c79-bf4d-74c3c0cc6728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.541857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ba7b6bb2-1664-4c79-bf4d-74c3c0cc6728 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:26.543196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ba7b6bb2-1664-4c79-bf4d-74c3c0cc6728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:26.551945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 680/1371] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:26.864060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e866d-2481-403c-a135-6c5bdf6f0c80 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.864060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e866d-2481-403c-a135-6c5bdf6f0c80 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.864060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e866d-2481-403c-a135-6c5bdf6f0c80 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.865347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 692/1372] 10.222.0.62 () {54 vars in 963 bytes} [Fri May 27 03:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:26.872679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbdcbb-25e8-4a1d-a2c2-659ee8f8194e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.872978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbdcbb-25e8-4a1d-a2c2-659ee8f8194e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.873264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbdcbb-25e8-4a1d-a2c2-659ee8f8194e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.879767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:26.924900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-51bbdcbb-25e8-4a1d-a2c2-659ee8f8194e None None] MFA Rules not processed for user `4a8b4111f8774b03815a525fdbdaa4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:26.926538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-51bbdcbb-25e8-4a1d-a2c2-659ee8f8194e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:26.994386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 681/1373] 10.222.0.62 () {58 vars in 1060 bytes} [Fri May 27 03:16:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:27.001898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e866d-2481-403c-a135-6c5bdf6f0c80 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:27.012834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e866d-2481-403c-a135-6c5bdf6f0c80 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:27.016458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ca8a7-4e61-4e17-83de-f7efa5546b49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:27.017191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca8a7-4e61-4e17-83de-f7efa5546b49 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:27.017552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca8a7-4e61-4e17-83de-f7efa5546b49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:27.017905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca8a7-4e61-4e17-83de-f7efa5546b49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:27.027580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ca8a7-4e61-4e17-83de-f7efa5546b49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:27.029128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ca8a7-4e61-4e17-83de-f7efa5546b49 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:27.039369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 693/1374] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:28.572182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19810-886c-4e88-8587-38d712d56de3 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:28.572769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19810-886c-4e88-8587-38d712d56de3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:28.572769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19810-886c-4e88-8587-38d712d56de3 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.574305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 682/1375] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:28.580194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2711f-a705-4de7-bafd-ca774183e2cf None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:28.580762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2711f-a705-4de7-bafd-ca774183e2cf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:28.580762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2711f-a705-4de7-bafd-ca774183e2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.601224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a1b2711f-a705-4de7-bafd-ca774183e2cf None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:28.602468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a1b2711f-a705-4de7-bafd-ca774183e2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:28.610315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 694/1376] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:28.655812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19810-886c-4e88-8587-38d712d56de3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:28.670771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19810-886c-4e88-8587-38d712d56de3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:28.676380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3d9f6-5ef8-4181-b971-637320ad7024 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:28.677054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3d9f6-5ef8-4181-b971-637320ad7024 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:28.677335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3d9f6-5ef8-4181-b971-637320ad7024 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:28.677613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3d9f6-5ef8-4181-b971-637320ad7024 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.687639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3d9f6-5ef8-4181-b971-637320ad7024 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:28.689324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3d9f6-5ef8-4181-b971-637320ad7024 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:28.700301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 683/1377] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:31.016579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469fa0-abeb-40fd-b2a9-3d07e4fda952 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:31.017129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469fa0-abeb-40fd-b2a9-3d07e4fda952 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:31.017129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469fa0-abeb-40fd-b2a9-3d07e4fda952 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:31.018503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 695/1378] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:31.024268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67df71-8986-41b9-a1cd-903e87a9b81c None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:31.025061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67df71-8986-41b9-a1cd-903e87a9b81c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:31.025344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67df71-8986-41b9-a1cd-903e87a9b81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:31.044027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2a67df71-8986-41b9-a1cd-903e87a9b81c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:31.045204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2a67df71-8986-41b9-a1cd-903e87a9b81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:31.053496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 684/1379] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:31.096645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36469fa0-abeb-40fd-b2a9-3d07e4fda952 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:31.111685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36469fa0-abeb-40fd-b2a9-3d07e4fda952 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:31.117960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5e580-68fb-4565-9115-17f6b313b5e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:31.118594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e580-68fb-4565-9115-17f6b313b5e7 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:31.118878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e580-68fb-4565-9115-17f6b313b5e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:31.119155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e580-68fb-4565-9115-17f6b313b5e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:31.128187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5e580-68fb-4565-9115-17f6b313b5e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:31.129757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5e580-68fb-4565-9115-17f6b313b5e7 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:31.140842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 696/1380] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:33.358016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2d1a0-04dd-47d4-87be-42c35d776a76 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:33.358543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2d1a0-04dd-47d4-87be-42c35d776a76 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:33.358543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2d1a0-04dd-47d4-87be-42c35d776a76 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:33.359971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 685/1381] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:33.365332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e1056-fa22-4d90-8f6a-bef70cabc335 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:33.365770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e1056-fa22-4d90-8f6a-bef70cabc335 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:33.365934 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e1056-fa22-4d90-8f6a-bef70cabc335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:33.383728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-438e1056-fa22-4d90-8f6a-bef70cabc335 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:33.385204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-438e1056-fa22-4d90-8f6a-bef70cabc335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:33.393113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 697/1382] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:33.435153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2d1a0-04dd-47d4-87be-42c35d776a76 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:33.450013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2d1a0-04dd-47d4-87be-42c35d776a76 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:33.455501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4efa8-4936-42ed-b74c-6dfe23fe02dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:33.456242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4efa8-4936-42ed-b74c-6dfe23fe02dc service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:33.456594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4efa8-4936-42ed-b74c-6dfe23fe02dc service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:33.456939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4efa8-4936-42ed-b74c-6dfe23fe02dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:33.466276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4efa8-4936-42ed-b74c-6dfe23fe02dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:33.467950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4efa8-4936-42ed-b74c-6dfe23fe02dc service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:33.478943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 686/1383] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:35.695365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c54b4-5c6d-4db5-92cc-cf3327510961 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:35.695956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c54b4-5c6d-4db5-92cc-cf3327510961 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:35.695956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c54b4-5c6d-4db5-92cc-cf3327510961 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:35.697631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 698/1384] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:35.702866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf351c8-fb3f-47a7-bb17-a3f9957db0d3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:35.703257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf351c8-fb3f-47a7-bb17-a3f9957db0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:35.703432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf351c8-fb3f-47a7-bb17-a3f9957db0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:35.723024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fcf351c8-fb3f-47a7-bb17-a3f9957db0d3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:35.724274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fcf351c8-fb3f-47a7-bb17-a3f9957db0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:35.732145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 687/1385] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:35.776060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c54b4-5c6d-4db5-92cc-cf3327510961 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:35.790867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c54b4-5c6d-4db5-92cc-cf3327510961 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:35.796465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048671fe-d8d0-4bff-aa30-4ce1d8aafd8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:35.797183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048671fe-d8d0-4bff-aa30-4ce1d8aafd8b service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:35.797551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048671fe-d8d0-4bff-aa30-4ce1d8aafd8b service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:35.797895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048671fe-d8d0-4bff-aa30-4ce1d8aafd8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:35.806607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048671fe-d8d0-4bff-aa30-4ce1d8aafd8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:35.808189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048671fe-d8d0-4bff-aa30-4ce1d8aafd8b service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:35.817807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 699/1386] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:38.045305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa859e-f100-4920-a68d-90f412152537 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:38.045830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa859e-f100-4920-a68d-90f412152537 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:38.045830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa859e-f100-4920-a68d-90f412152537 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:38.047161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 688/1387] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:38.052628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa594fd-5418-4fd9-bce0-d98651e539bb None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:38.053177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa594fd-5418-4fd9-bce0-d98651e539bb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:38.053177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa594fd-5418-4fd9-bce0-d98651e539bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:38.071898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-8fa594fd-5418-4fd9-bce0-d98651e539bb None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:38.073085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8fa594fd-5418-4fd9-bce0-d98651e539bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:38.081036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 700/1388] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:38.129868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa859e-f100-4920-a68d-90f412152537 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:38.151149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa859e-f100-4920-a68d-90f412152537 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:38.159124 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefce48-ae91-4499-bca2-e60743592be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:38.160063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefce48-ae91-4499-bca2-e60743592be7 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:38.160450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefce48-ae91-4499-bca2-e60743592be7 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:38.160817 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefce48-ae91-4499-bca2-e60743592be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:38.172154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefce48-ae91-4499-bca2-e60743592be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:38.174380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefce48-ae91-4499-bca2-e60743592be7 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:38.187145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 689/1389] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:40.396410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ffea7-a3a8-4503-982a-1981892cd66a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:40.397717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ffea7-a3a8-4503-982a-1981892cd66a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:40.397717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ffea7-a3a8-4503-982a-1981892cd66a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:40.398874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 701/1390] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:40.404843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb932b-10a4-40e6-92fc-3c35169fad77 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:40.405267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb932b-10a4-40e6-92fc-3c35169fad77 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:40.405560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb932b-10a4-40e6-92fc-3c35169fad77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:40.428095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-58cb932b-10a4-40e6-92fc-3c35169fad77 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:40.429604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-58cb932b-10a4-40e6-92fc-3c35169fad77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:40.439771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 690/1391] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:40.502695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ffea7-a3a8-4503-982a-1981892cd66a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:40.517473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ffea7-a3a8-4503-982a-1981892cd66a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:40.523937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd61b4b-e896-411c-8fff-76d4e0b08760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:40.524580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd61b4b-e896-411c-8fff-76d4e0b08760 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:40.524867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd61b4b-e896-411c-8fff-76d4e0b08760 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:40.525152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd61b4b-e896-411c-8fff-76d4e0b08760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:40.533758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd61b4b-e896-411c-8fff-76d4e0b08760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:40.535350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd61b4b-e896-411c-8fff-76d4e0b08760 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:40.544756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 702/1392] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:42.686744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a8463-ffb1-4787-9c45-e495936b1304 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.687300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a8463-ffb1-4787-9c45-e495936b1304 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.687300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a8463-ffb1-4787-9c45-e495936b1304 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.688682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 691/1393] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:42.693820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99f6b6-5f3d-4d3c-92f0-11a7210c39f9 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.694636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99f6b6-5f3d-4d3c-92f0-11a7210c39f9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.694921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99f6b6-5f3d-4d3c-92f0-11a7210c39f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.712771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-dd99f6b6-5f3d-4d3c-92f0-11a7210c39f9 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:42.713937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-dd99f6b6-5f3d-4d3c-92f0-11a7210c39f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:42.721639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 703/1394] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:42.767882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f56e8-9a9c-4780-8bb4-59ac612cf877 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.768194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f56e8-9a9c-4780-8bb4-59ac612cf877 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.768471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f56e8-9a9c-4780-8bb4-59ac612cf877 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.769836 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 692/1395] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:42.775091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15674-2ea7-428c-929a-a678aada477a None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.775381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15674-2ea7-428c-929a-a678aada477a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.775662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15674-2ea7-428c-929a-a678aada477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.777039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8ea1-ac97-4095-b636-2cc610d562cc None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.777395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8ea1-ac97-4095-b636-2cc610d562cc None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.777733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8ea1-ac97-4095-b636-2cc610d562cc None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.779057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 693/1396] 10.222.0.62 () {54 vars in 962 bytes} [Fri May 27 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:42.783903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4ffb-0c94-4362-82d8-2f4a8f9873ba None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.784186 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4ffb-0c94-4362-82d8-2f4a8f9873ba None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.784460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4ffb-0c94-4362-82d8-2f4a8f9873ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.794371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c1e15674-2ea7-428c-929a-a678aada477a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:42.795550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c1e15674-2ea7-428c-929a-a678aada477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:42.804134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 704/1397] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:16:42.832426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d83c4ffb-0c94-4362-82d8-2f4a8f9873ba None None] MFA Rules not processed for user `7be3a546aa644281a0d3594395bbdf33`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:42.834117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d83c4ffb-0c94-4362-82d8-2f4a8f9873ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:42.848433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e15674-2ea7-428c-929a-a678aada477a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:42.861175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e15674-2ea7-428c-929a-a678aada477a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:42.866656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:42.867305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.867592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.867915 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.876141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:42.877734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:42.887218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 705/1398] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:16:42.896291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:42.909284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 694/1399] 10.222.0.62 () {58 vars in 1059 bytes} [Fri May 27 03:16:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:16:42.911037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:42.927056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4e5f-9601-48a6-b502-180f65934be0 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:42.933850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dc09d-1a63-4743-a1bd-918dbd34b1a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7be3a546aa644281a0d3594395bbdf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:42.933850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc09d-1a63-4743-a1bd-918dbd34b1a6 service glance] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.933850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc09d-1a63-4743-a1bd-918dbd34b1a6 service glance] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.933850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc09d-1a63-4743-a1bd-918dbd34b1a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.947271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dc09d-1a63-4743-a1bd-918dbd34b1a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:42.948744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dc09d-1a63-4743-a1bd-918dbd34b1a6 service glance] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:42.958677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 706/1400] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:16:43.487137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e384c0a-de15-4c30-850f-cbb556769720 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:43.487428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e384c0a-de15-4c30-850f-cbb556769720 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:43.487709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e384c0a-de15-4c30-850f-cbb556769720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:43.537564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8e384c0a-de15-4c30-850f-cbb556769720 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:43.539297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8e384c0a-de15-4c30-850f-cbb556769720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:43.603649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 695/1401] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:16:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:43.647542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205236e2-0896-44b2-8c70-dc4f62cbbe6c None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:43.647964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205236e2-0896-44b2-8c70-dc4f62cbbe6c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:43.648342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205236e2-0896-44b2-8c70-dc4f62cbbe6c None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:43.649854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 707/1402] 10.222.0.62 () {54 vars in 942 bytes} [Fri May 27 03:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:43.656265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637b3a-251b-4036-b40f-09e690dac7ee None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:43.656560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637b3a-251b-4036-b40f-09e690dac7ee None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:43.656847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637b3a-251b-4036-b40f-09e690dac7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:43.704494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-49637b3a-251b-4036-b40f-09e690dac7ee None None] MFA Rules not processed for user `4b633648662a4b51bad083fd4637f204`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:43.705890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-49637b3a-251b-4036-b40f-09e690dac7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:43.777658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 696/1403] 10.222.0.62 () {58 vars in 1039 bytes} [Fri May 27 03:16:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:43.786965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205236e2-0896-44b2-8c70-dc4f62cbbe6c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:43.794975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=70 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:43.801735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205236e2-0896-44b2-8c70-dc4f62cbbe6c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:43.806685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b8b01-0906-4de1-9399-71fc6cda6e88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:43.807321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b8b01-0906-4de1-9399-71fc6cda6e88 service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:43.807609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b8b01-0906-4de1-9399-71fc6cda6e88 service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:43.807919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b8b01-0906-4de1-9399-71fc6cda6e88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:43.816095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b8b01-0906-4de1-9399-71fc6cda6e88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:43.817669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b8b01-0906-4de1-9399-71fc6cda6e88 service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:43.825833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 708/1404] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:16:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:16:45.136612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706d7f1-8496-48aa-8b7d-eb2ba87aaf78 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.137180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706d7f1-8496-48aa-8b7d-eb2ba87aaf78 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.137180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706d7f1-8496-48aa-8b7d-eb2ba87aaf78 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.138532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 697/1405] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:45.143859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b66cbf-ed66-43b4-83c3-78e7ba4b56a6 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.144316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b66cbf-ed66-43b4-83c3-78e7ba4b56a6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.144422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b66cbf-ed66-43b4-83c3-78e7ba4b56a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.164440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b7b66cbf-ed66-43b4-83c3-78e7ba4b56a6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:45.167155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b7b66cbf-ed66-43b4-83c3-78e7ba4b56a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:45.175355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 709/1406] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:45.222597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706d7f1-8496-48aa-8b7d-eb2ba87aaf78 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:45.238731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706d7f1-8496-48aa-8b7d-eb2ba87aaf78 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:45.244456 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a47e0-3fd9-4383-bf26-991a1d3c4b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:45.245299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a47e0-3fd9-4383-bf26-991a1d3c4b13 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.245794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a47e0-3fd9-4383-bf26-991a1d3c4b13 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.246066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a47e0-3fd9-4383-bf26-991a1d3c4b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.254566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a47e0-3fd9-4383-bf26-991a1d3c4b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:45.256264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a47e0-3fd9-4383-bf26-991a1d3c4b13 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:45.266210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 698/1407] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:55.157743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=8 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:55.261957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:07.157680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2110a-6bad-48ed-b2d8-2ee8383dc594 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:07.158322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2110a-6bad-48ed-b2d8-2ee8383dc594 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:07.158322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2110a-6bad-48ed-b2d8-2ee8383dc594 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:07.159604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 710/1408] 10.222.0.62 () {54 vars in 918 bytes} [Fri May 27 03:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:07.165225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2afe-6179-4341-a149-ece4f2b96064 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:07.165719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2afe-6179-4341-a149-ece4f2b96064 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:07.165838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2afe-6179-4341-a149-ece4f2b96064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:07.183298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e29b2afe-6179-4341-a149-ece4f2b96064 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:07.184510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e29b2afe-6179-4341-a149-ece4f2b96064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:07.192790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 699/1409] 10.222.0.62 () {58 vars in 1015 bytes} [Fri May 27 03:17:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:07.206162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae3d3-b9c0-432a-b955-26409cfbe9b1 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:07.206458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae3d3-b9c0-432a-b955-26409cfbe9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:07.206742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ae3d3-b9c0-432a-b955-26409cfbe9b1 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:07.208228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 711/1410] 10.222.0.62 () {54 vars in 918 bytes} [Fri May 27 03:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:07.213649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4871e0e-a7dc-42c5-bb46-7b80f2158dfe None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:07.213932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4871e0e-a7dc-42c5-bb46-7b80f2158dfe None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:07.214232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4871e0e-a7dc-42c5-bb46-7b80f2158dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:07.232953 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f4871e0e-a7dc-42c5-bb46-7b80f2158dfe None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:07.234118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f4871e0e-a7dc-42c5-bb46-7b80f2158dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:07.243046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 700/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Fri May 27 03:17:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:09.916346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1f1e4-bac2-41ff-b5f4-973a4c94cbbd None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.916927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1f1e4-bac2-41ff-b5f4-973a4c94cbbd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.916927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1f1e4-bac2-41ff-b5f4-973a4c94cbbd None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.918280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 712/1412] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:09.923563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32629b67-f334-4651-b254-b17095be1fc7 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.924000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32629b67-f334-4651-b254-b17095be1fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.924147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32629b67-f334-4651-b254-b17095be1fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.941921 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-32629b67-f334-4651-b254-b17095be1fc7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:09.943092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-32629b67-f334-4651-b254-b17095be1fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:09.951119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 701/1413] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:09.994305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1f1e4-bac2-41ff-b5f4-973a4c94cbbd None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:10.009092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1f1e4-bac2-41ff-b5f4-973a4c94cbbd None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:10.014444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10943b8-ba48-4c75-9f68-b1cc1faff1dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:10.015078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10943b8-ba48-4c75-9f68-b1cc1faff1dc service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:10.015362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10943b8-ba48-4c75-9f68-b1cc1faff1dc service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:10.015636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10943b8-ba48-4c75-9f68-b1cc1faff1dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:10.024221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10943b8-ba48-4c75-9f68-b1cc1faff1dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:10.025776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10943b8-ba48-4c75-9f68-b1cc1faff1dc service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:10.034855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 713/1414] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:10.999453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c471e-f232-40fc-b65d-36592fd2e4c0 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:11.000033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c471e-f232-40fc-b65d-36592fd2e4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:11.000033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c471e-f232-40fc-b65d-36592fd2e4c0 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:11.001415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 702/1415] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:11.006270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae2797-4acf-4f64-a4a5-a0f3555b68e8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:11.006700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae2797-4acf-4f64-a4a5-a0f3555b68e8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:11.006864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae2797-4acf-4f64-a4a5-a0f3555b68e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:11.025554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-1aae2797-4acf-4f64-a4a5-a0f3555b68e8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:11.026731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1aae2797-4acf-4f64-a4a5-a0f3555b68e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:11.034355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 714/1416] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:11.200576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c471e-f232-40fc-b65d-36592fd2e4c0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:11.215301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c471e-f232-40fc-b65d-36592fd2e4c0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:11.220622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecc709-53eb-4b50-9db4-2a6a0e2cb91d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:11.221251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecc709-53eb-4b50-9db4-2a6a0e2cb91d service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:11.221533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecc709-53eb-4b50-9db4-2a6a0e2cb91d service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:11.221811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecc709-53eb-4b50-9db4-2a6a0e2cb91d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:11.229729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecc709-53eb-4b50-9db4-2a6a0e2cb91d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:11.231312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecc709-53eb-4b50-9db4-2a6a0e2cb91d service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:11.240630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 703/1417] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:12.153763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed18d1-527e-4895-8644-508aa0e3e7c9 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.154342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed18d1-527e-4895-8644-508aa0e3e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.154342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed18d1-527e-4895-8644-508aa0e3e7c9 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.155771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 715/1418] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:12.160738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d436d2-4636-45ec-a6e3-698dc017333a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.161172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d436d2-4636-45ec-a6e3-698dc017333a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.161279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d436d2-4636-45ec-a6e3-698dc017333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.179576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-44d436d2-4636-45ec-a6e3-698dc017333a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:12.180929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-44d436d2-4636-45ec-a6e3-698dc017333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:12.188916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 704/1419] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:12.338775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed18d1-527e-4895-8644-508aa0e3e7c9 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:12.353170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed18d1-527e-4895-8644-508aa0e3e7c9 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:12.358448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273715c-f00f-4698-a03f-fb0db190d7be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:12.359072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273715c-f00f-4698-a03f-fb0db190d7be service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.359356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273715c-f00f-4698-a03f-fb0db190d7be service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.359633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273715c-f00f-4698-a03f-fb0db190d7be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.367469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273715c-f00f-4698-a03f-fb0db190d7be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:12.369074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273715c-f00f-4698-a03f-fb0db190d7be service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:12.378102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 716/1420] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:13.283454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210db888-a96c-4737-936b-5277b0a4a1c7 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:13.284023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210db888-a96c-4737-936b-5277b0a4a1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:13.284023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210db888-a96c-4737-936b-5277b0a4a1c7 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:13.285315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 705/1421] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:13.290633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462552c1-54a3-442a-afe2-799b6d4fb3cf None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:13.291061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462552c1-54a3-442a-afe2-799b6d4fb3cf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:13.291170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462552c1-54a3-442a-afe2-799b6d4fb3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:13.309671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-462552c1-54a3-442a-afe2-799b6d4fb3cf None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:13.310856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-462552c1-54a3-442a-afe2-799b6d4fb3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:13.318494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 717/1422] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:13.465938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210db888-a96c-4737-936b-5277b0a4a1c7 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:13.480643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210db888-a96c-4737-936b-5277b0a4a1c7 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:13.485919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663e361-510b-449f-b3e9-674badf45a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:13.486542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663e361-510b-449f-b3e9-674badf45a2e service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:13.486828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663e361-510b-449f-b3e9-674badf45a2e service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:13.487100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663e361-510b-449f-b3e9-674badf45a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:13.496346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663e361-510b-449f-b3e9-674badf45a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:13.497919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663e361-510b-449f-b3e9-674badf45a2e service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:13.507564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 706/1423] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:14.436133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d86399-fb9a-418a-87fe-0944195a1720 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:14.436698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d86399-fb9a-418a-87fe-0944195a1720 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:14.436698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d86399-fb9a-418a-87fe-0944195a1720 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:14.438069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 718/1424] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:14.443034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4e0b-157e-4946-a49c-6ef4c378b53b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:14.443467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4e0b-157e-4946-a49c-6ef4c378b53b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:14.443590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4e0b-157e-4946-a49c-6ef4c378b53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:14.462141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d00d4e0b-157e-4946-a49c-6ef4c378b53b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:14.463322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d00d4e0b-157e-4946-a49c-6ef4c378b53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:14.471206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 707/1425] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:14.646010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d86399-fb9a-418a-87fe-0944195a1720 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:14.660083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d86399-fb9a-418a-87fe-0944195a1720 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:14.665448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d51321-fb28-4b92-94f2-cfea1d487b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:14.666082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d51321-fb28-4b92-94f2-cfea1d487b50 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:14.666371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d51321-fb28-4b92-94f2-cfea1d487b50 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:14.666654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d51321-fb28-4b92-94f2-cfea1d487b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:14.674651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d51321-fb28-4b92-94f2-cfea1d487b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:14.676273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d51321-fb28-4b92-94f2-cfea1d487b50 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:14.685985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 719/1426] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:15.532436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c006c-9efa-4cbf-9b29-56f75a085402 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:15.533004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c006c-9efa-4cbf-9b29-56f75a085402 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:15.533004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c006c-9efa-4cbf-9b29-56f75a085402 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:15.534365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 708/1427] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:15.539361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37d44f-128f-43ee-82ce-7c7002718fc7 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:15.539752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37d44f-128f-43ee-82ce-7c7002718fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:15.539991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37d44f-128f-43ee-82ce-7c7002718fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:15.557503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9b37d44f-128f-43ee-82ce-7c7002718fc7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:15.558625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9b37d44f-128f-43ee-82ce-7c7002718fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:15.566315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 720/1428] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:15.715620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c006c-9efa-4cbf-9b29-56f75a085402 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:15.729454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c006c-9efa-4cbf-9b29-56f75a085402 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:15.734685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63620933-1848-42c9-86a1-16433838f6cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:15.735312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620933-1848-42c9-86a1-16433838f6cc service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:15.735594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620933-1848-42c9-86a1-16433838f6cc service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:15.735894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620933-1848-42c9-86a1-16433838f6cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:15.743701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63620933-1848-42c9-86a1-16433838f6cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:15.745474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63620933-1848-42c9-86a1-16433838f6cc service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:15.754962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 709/1429] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:16.595429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955f3c-02fa-489a-ad63-2cc6dcfbddfd None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:16.596045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955f3c-02fa-489a-ad63-2cc6dcfbddfd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:16.596045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955f3c-02fa-489a-ad63-2cc6dcfbddfd None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:16.597393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 721/1430] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:16.602896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f47a28-bb08-4f84-b31f-5522a6f1c509 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:16.603329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f47a28-bb08-4f84-b31f-5522a6f1c509 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:16.603465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f47a28-bb08-4f84-b31f-5522a6f1c509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:16.621927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-01f47a28-bb08-4f84-b31f-5522a6f1c509 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:16.623141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-01f47a28-bb08-4f84-b31f-5522a6f1c509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:16.630846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 710/1431] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:16.779583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2955f3c-02fa-489a-ad63-2cc6dcfbddfd None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:16.794087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2955f3c-02fa-489a-ad63-2cc6dcfbddfd None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:16.799344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76540243-a910-4592-aca1-d2fab0cde5cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:16.800004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540243-a910-4592-aca1-d2fab0cde5cf service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:16.800289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540243-a910-4592-aca1-d2fab0cde5cf service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:16.800565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76540243-a910-4592-aca1-d2fab0cde5cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:16.808381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76540243-a910-4592-aca1-d2fab0cde5cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:16.809933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76540243-a910-4592-aca1-d2fab0cde5cf service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:16.818909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 722/1432] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:17.662124 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a09c0-c5fd-4fbe-a1d6-625d4598d095 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:17.662671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a09c0-c5fd-4fbe-a1d6-625d4598d095 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:17.662671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a09c0-c5fd-4fbe-a1d6-625d4598d095 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:17.664109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 711/1433] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:17.669095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496184f-c1b1-42a7-a1f2-abfbb609ea50 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:17.669532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496184f-c1b1-42a7-a1f2-abfbb609ea50 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:17.669654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496184f-c1b1-42a7-a1f2-abfbb609ea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:17.688320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-8496184f-c1b1-42a7-a1f2-abfbb609ea50 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:17.689496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8496184f-c1b1-42a7-a1f2-abfbb609ea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:17.697683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 723/1434] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:17.845479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a09c0-c5fd-4fbe-a1d6-625d4598d095 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:17.860241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a09c0-c5fd-4fbe-a1d6-625d4598d095 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:17.865770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5c33a-3e2b-473a-94a5-6edced00193e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:17.866428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5c33a-3e2b-473a-94a5-6edced00193e service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:17.867414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5c33a-3e2b-473a-94a5-6edced00193e service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:17.867696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5c33a-3e2b-473a-94a5-6edced00193e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:17.875937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5c33a-3e2b-473a-94a5-6edced00193e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:17.877623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5c33a-3e2b-473a-94a5-6edced00193e service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:17.886807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 712/1435] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:18.742885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde934bd-5046-4a30-ae58-903552a3b843 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:18.743496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde934bd-5046-4a30-ae58-903552a3b843 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:18.743496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde934bd-5046-4a30-ae58-903552a3b843 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:18.744957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 724/1436] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:18.750155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74ea14-95e6-4d17-9b54-9fa9b1c3e849 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:18.750565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74ea14-95e6-4d17-9b54-9fa9b1c3e849 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:18.750710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74ea14-95e6-4d17-9b54-9fa9b1c3e849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:18.769039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9e74ea14-95e6-4d17-9b54-9fa9b1c3e849 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:18.770230 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9e74ea14-95e6-4d17-9b54-9fa9b1c3e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:18.778442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 713/1437] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:18.931937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde934bd-5046-4a30-ae58-903552a3b843 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:18.947159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde934bd-5046-4a30-ae58-903552a3b843 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:18.952626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b766dfe-5296-4b1c-b049-24165bc41036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:18.953270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766dfe-5296-4b1c-b049-24165bc41036 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:18.953587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766dfe-5296-4b1c-b049-24165bc41036 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:18.953867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766dfe-5296-4b1c-b049-24165bc41036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:18.962308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b766dfe-5296-4b1c-b049-24165bc41036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:18.963908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b766dfe-5296-4b1c-b049-24165bc41036 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:18.973185 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 725/1438] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:21.044972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948f24a-f225-410a-8fcf-f25697824a6f None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:21.045537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948f24a-f225-410a-8fcf-f25697824a6f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:21.045537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948f24a-f225-410a-8fcf-f25697824a6f None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:21.046899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 714/1439] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:21.052080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de4b2-951b-4d3b-9b9d-d019826f1d20 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:21.052557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de4b2-951b-4d3b-9b9d-d019826f1d20 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:21.052655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de4b2-951b-4d3b-9b9d-d019826f1d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:21.070626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-cb6de4b2-951b-4d3b-9b9d-d019826f1d20 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:21.071843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-cb6de4b2-951b-4d3b-9b9d-d019826f1d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:21.079557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 726/1440] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:21.118523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948f24a-f225-410a-8fcf-f25697824a6f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:21.129228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948f24a-f225-410a-8fcf-f25697824a6f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:21.132794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdedbaf-8e91-4abb-8b21-16204acac53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:21.133429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdedbaf-8e91-4abb-8b21-16204acac53c service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:21.133718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdedbaf-8e91-4abb-8b21-16204acac53c service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:21.134113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdedbaf-8e91-4abb-8b21-16204acac53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:21.142937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdedbaf-8e91-4abb-8b21-16204acac53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:21.144430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdedbaf-8e91-4abb-8b21-16204acac53c service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:21.154119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 715/1441] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:23.484969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3f301-2447-47c7-8812-f2b75edcbdeb None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:23.485537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3f301-2447-47c7-8812-f2b75edcbdeb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:23.485537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3f301-2447-47c7-8812-f2b75edcbdeb None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:23.486923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 727/1442] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:23.492381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694a714-0047-4479-a320-76a5243cf576 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:23.492822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694a714-0047-4479-a320-76a5243cf576 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:23.492930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694a714-0047-4479-a320-76a5243cf576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:23.510542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-7694a714-0047-4479-a320-76a5243cf576 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:23.511713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7694a714-0047-4479-a320-76a5243cf576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:23.519531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 716/1443] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:23.557305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3f301-2447-47c7-8812-f2b75edcbdeb None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:23.567778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3f301-2447-47c7-8812-f2b75edcbdeb None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:23.571172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c62435-adce-44a4-bf86-e9955d6b0ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:23.571796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c62435-adce-44a4-bf86-e9955d6b0ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:23.572106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c62435-adce-44a4-bf86-e9955d6b0ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:23.572382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c62435-adce-44a4-bf86-e9955d6b0ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:23.580887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c62435-adce-44a4-bf86-e9955d6b0ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:23.582296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c62435-adce-44a4-bf86-e9955d6b0ca4 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:23.592064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 728/1444] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:26.071894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eca74-2f8c-46e6-bd06-25610bc4508f None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:26.072453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eca74-2f8c-46e6-bd06-25610bc4508f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:26.072453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eca74-2f8c-46e6-bd06-25610bc4508f None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:26.073813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 717/1445] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:26.079051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4c6c7-af50-4f23-9e8d-682d74666a4e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:26.079480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4c6c7-af50-4f23-9e8d-682d74666a4e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:26.079635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4c6c7-af50-4f23-9e8d-682d74666a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:26.097762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-19c4c6c7-af50-4f23-9e8d-682d74666a4e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:26.098944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-19c4c6c7-af50-4f23-9e8d-682d74666a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:26.106588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 729/1446] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:26.144293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eca74-2f8c-46e6-bd06-25610bc4508f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:26.155374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eca74-2f8c-46e6-bd06-25610bc4508f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:26.158847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd99f6d-7722-469a-9d83-1518f008a7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:26.159471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99f6d-7722-469a-9d83-1518f008a7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:26.159752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99f6d-7722-469a-9d83-1518f008a7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:26.160071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99f6d-7722-469a-9d83-1518f008a7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:26.168674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd99f6d-7722-469a-9d83-1518f008a7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:26.170089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd99f6d-7722-469a-9d83-1518f008a7a7 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:26.179352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 718/1447] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:28.485912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9d408-7dee-4330-9356-1cce03fe75b4 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:28.487695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9d408-7dee-4330-9356-1cce03fe75b4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:28.487695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9d408-7dee-4330-9356-1cce03fe75b4 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:28.487937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 730/1448] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:28.493972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f77a1-0b6d-47eb-b855-99fbd78ec1dd None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:28.494428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f77a1-0b6d-47eb-b855-99fbd78ec1dd None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:28.494536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f77a1-0b6d-47eb-b855-99fbd78ec1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:28.511745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ec1f77a1-0b6d-47eb-b855-99fbd78ec1dd None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:28.512951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ec1f77a1-0b6d-47eb-b855-99fbd78ec1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:28.520667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 719/1449] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:28.559322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9d408-7dee-4330-9356-1cce03fe75b4 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:28.569971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9d408-7dee-4330-9356-1cce03fe75b4 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:28.573787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843d4f-50f5-48c6-ac27-8216fa59b8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:28.574414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843d4f-50f5-48c6-ac27-8216fa59b8ca service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:28.574696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843d4f-50f5-48c6-ac27-8216fa59b8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:28.574971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843d4f-50f5-48c6-ac27-8216fa59b8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:28.583640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c843d4f-50f5-48c6-ac27-8216fa59b8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:28.585856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c843d4f-50f5-48c6-ac27-8216fa59b8ca service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:28.595101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 731/1450] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:30.842151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ceb9c-9552-4022-95de-d4420df5bfd3 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:30.842720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ceb9c-9552-4022-95de-d4420df5bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:30.842720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ceb9c-9552-4022-95de-d4420df5bfd3 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:30.844099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 720/1451] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:30.849528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ad64c-6e4b-491b-9a4b-381b43b47eeb None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:30.849986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ad64c-6e4b-491b-9a4b-381b43b47eeb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:30.850095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ad64c-6e4b-491b-9a4b-381b43b47eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:30.868113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-e80ad64c-6e4b-491b-9a4b-381b43b47eeb None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:30.869326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e80ad64c-6e4b-491b-9a4b-381b43b47eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:30.876901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 732/1452] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:30.929688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285529-64c8-44aa-a948-233d75fc4612 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:30.929977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285529-64c8-44aa-a948-233d75fc4612 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:30.930256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285529-64c8-44aa-a948-233d75fc4612 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:30.931632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 721/1453] 10.222.0.62 () {54 vars in 964 bytes} [Fri May 27 03:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:30.937780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bb167-aeeb-49c8-ba16-9381535c6eff None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:30.938069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bb167-aeeb-49c8-ba16-9381535c6eff None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:30.938349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bb167-aeeb-49c8-ba16-9381535c6eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:30.974748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f52bb167-aeeb-49c8-ba16-9381535c6eff None None] MFA Rules not processed for user `a364b6179abb4ad0883898090acef827`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:30.976667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f52bb167-aeeb-49c8-ba16-9381535c6eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:30.983106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 733/1454] 10.222.0.62 () {58 vars in 1061 bytes} [Fri May 27 03:17:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:30.992552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285529-64c8-44aa-a948-233d75fc4612 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:31.003429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285529-64c8-44aa-a948-233d75fc4612 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:31.006859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59feb273-3adf-415b-9b83-26d6b3e8e071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:31.007485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feb273-3adf-415b-9b83-26d6b3e8e071 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:31.007764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feb273-3adf-415b-9b83-26d6b3e8e071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:31.008070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feb273-3adf-415b-9b83-26d6b3e8e071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:31.016820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59feb273-3adf-415b-9b83-26d6b3e8e071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:31.018237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59feb273-3adf-415b-9b83-26d6b3e8e071 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:31.027846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 722/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:33.827598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aedb2-eaa8-405a-92ad-e4bfbb627b07 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:33.828198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aedb2-eaa8-405a-92ad-e4bfbb627b07 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:33.828198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aedb2-eaa8-405a-92ad-e4bfbb627b07 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:33.829560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:33.835222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa762ad0-12c3-4365-a2b3-88532c9a24f2 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:33.836177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa762ad0-12c3-4365-a2b3-88532c9a24f2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:33.836277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa762ad0-12c3-4365-a2b3-88532c9a24f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:33.853950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fa762ad0-12c3-4365-a2b3-88532c9a24f2 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:33.855112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fa762ad0-12c3-4365-a2b3-88532c9a24f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:33.862675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:33.899693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aedb2-eaa8-405a-92ad-e4bfbb627b07 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:33.910145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aedb2-eaa8-405a-92ad-e4bfbb627b07 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:33.914446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43225faf-7a6d-457c-a333-008f8216e1df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:33.915085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43225faf-7a6d-457c-a333-008f8216e1df service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:33.915371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43225faf-7a6d-457c-a333-008f8216e1df service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:33.915653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43225faf-7a6d-457c-a333-008f8216e1df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:33.924676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43225faf-7a6d-457c-a333-008f8216e1df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:33.926092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43225faf-7a6d-457c-a333-008f8216e1df service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:33.935413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 735/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:36.345145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5b26-8902-49fd-8f4a-482a6cdaeb9a None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:36.345695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5b26-8902-49fd-8f4a-482a6cdaeb9a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:36.345822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5b26-8902-49fd-8f4a-482a6cdaeb9a None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:36.347081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:36.353147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa5c6-220c-41e4-94ad-1bb9809cca72 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:36.353559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa5c6-220c-41e4-94ad-1bb9809cca72 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:36.353703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa5c6-220c-41e4-94ad-1bb9809cca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:36.359616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=79 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:36.372255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-707aa5c6-220c-41e4-94ad-1bb9809cca72 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:36.373429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-707aa5c6-220c-41e4-94ad-1bb9809cca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:36.381255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:17:36.417978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd5b26-8902-49fd-8f4a-482a6cdaeb9a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:36.423323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=73 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:36.429595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd5b26-8902-49fd-8f4a-482a6cdaeb9a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:36.433075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4adbbf1-39b0-42fd-bbd6-48d0e70ff341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:36.433719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adbbf1-39b0-42fd-bbd6-48d0e70ff341 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:36.434005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adbbf1-39b0-42fd-bbd6-48d0e70ff341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:36.434285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adbbf1-39b0-42fd-bbd6-48d0e70ff341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:36.442940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adbbf1-39b0-42fd-bbd6-48d0e70ff341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:36.444406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adbbf1-39b0-42fd-bbd6-48d0e70ff341 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:36.453498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 725/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:38.689564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1463-08ea-42aa-a8d5-4b56edf733c7 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:38.690140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1463-08ea-42aa-a8d5-4b56edf733c7 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:38.690140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1463-08ea-42aa-a8d5-4b56edf733c7 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:38.691515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:38.697569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e656ce-f494-447c-bb19-5f3ba515e972 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:38.697975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e656ce-f494-447c-bb19-5f3ba515e972 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:38.698154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e656ce-f494-447c-bb19-5f3ba515e972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:38.715834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-93e656ce-f494-447c-bb19-5f3ba515e972 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:38.717033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-93e656ce-f494-447c-bb19-5f3ba515e972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:38.724805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:38.762656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a1463-08ea-42aa-a8d5-4b56edf733c7 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:38.773007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a1463-08ea-42aa-a8d5-4b56edf733c7 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:38.777150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff534f-c2bc-4325-b52c-5be2b330ee87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:38.777991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff534f-c2bc-4325-b52c-5be2b330ee87 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:38.778426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff534f-c2bc-4325-b52c-5be2b330ee87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:38.778738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff534f-c2bc-4325-b52c-5be2b330ee87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:38.787478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff534f-c2bc-4325-b52c-5be2b330ee87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:38.789453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff534f-c2bc-4325-b52c-5be2b330ee87 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:38.799429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 738/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:40.995454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5536d4-7707-404f-b4b7-af5867155d9a None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:40.995998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5536d4-7707-404f-b4b7-af5867155d9a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:40.996107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5536d4-7707-404f-b4b7-af5867155d9a None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:40.997444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:41.006731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc4d73-892e-4f55-8565-50d2092884ef None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:41.007127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc4d73-892e-4f55-8565-50d2092884ef None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:41.007320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc4d73-892e-4f55-8565-50d2092884ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:41.029262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-dbbc4d73-892e-4f55-8565-50d2092884ef None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:41.030478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-dbbc4d73-892e-4f55-8565-50d2092884ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:41.038085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:41.076025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5536d4-7707-404f-b4b7-af5867155d9a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:41.086860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5536d4-7707-404f-b4b7-af5867155d9a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:41.090380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7fd13-6314-46bc-8463-16cce63ee3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:41.091013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fd13-6314-46bc-8463-16cce63ee3bf service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:41.091299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fd13-6314-46bc-8463-16cce63ee3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:41.091574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fd13-6314-46bc-8463-16cce63ee3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:41.100331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7fd13-6314-46bc-8463-16cce63ee3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:41.102245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7fd13-6314-46bc-8463-16cce63ee3bf service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:41.111366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 728/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:43.364302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875094-1a94-4d98-9742-52770bc5d056 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:43.364764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875094-1a94-4d98-9742-52770bc5d056 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:43.364878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875094-1a94-4d98-9742-52770bc5d056 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:43.366288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:43.372133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb138c-1194-46fa-b106-10f46c907c6c None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:43.372539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb138c-1194-46fa-b106-10f46c907c6c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:43.372678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb138c-1194-46fa-b106-10f46c907c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:43.390192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-aeeb138c-1194-46fa-b106-10f46c907c6c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:43.391413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-aeeb138c-1194-46fa-b106-10f46c907c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:43.399166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:43.435223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875094-1a94-4d98-9742-52770bc5d056 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:43.445828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875094-1a94-4d98-9742-52770bc5d056 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:43.449361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f27f7-5147-4f6b-8d65-f58fe0209f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:43.449996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f27f7-5147-4f6b-8d65-f58fe0209f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:43.450283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f27f7-5147-4f6b-8d65-f58fe0209f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:43.450565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f27f7-5147-4f6b-8d65-f58fe0209f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:43.459286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f27f7-5147-4f6b-8d65-f58fe0209f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:43.461010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f27f7-5147-4f6b-8d65-f58fe0209f0d service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:43.470848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 741/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:53.384970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=17 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:53.466716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=17 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:16.438352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e776-e9b6-4000-a777-fb32354fde46 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:16.438956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e776-e9b6-4000-a777-fb32354fde46 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:16.438956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e776-e9b6-4000-a777-fb32354fde46 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:16.440313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 730/1471] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:16.445379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18b5e2-da15-4dc0-95fc-5e0c932b8040 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:16.445810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18b5e2-da15-4dc0-95fc-5e0c932b8040 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:16.445921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18b5e2-da15-4dc0-95fc-5e0c932b8040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:16.463387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-8f18b5e2-da15-4dc0-95fc-5e0c932b8040 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:16.464588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8f18b5e2-da15-4dc0-95fc-5e0c932b8040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:16.472154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 742/1472] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:16.518265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e5d42-eb1c-491f-98c0-bb377d0a3d6d None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:16.518557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e5d42-eb1c-491f-98c0-bb377d0a3d6d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:16.518839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e5d42-eb1c-491f-98c0-bb377d0a3d6d None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:16.520215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 731/1473] 10.222.0.62 () {54 vars in 962 bytes} [Fri May 27 03:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:16.524818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f979d-1046-461f-aed3-1bd2c89bd339 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:16.525109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f979d-1046-461f-aed3-1bd2c89bd339 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:16.525385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f979d-1046-461f-aed3-1bd2c89bd339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:16.543242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d16f979d-1046-461f-aed3-1bd2c89bd339 None None] MFA Rules not processed for user `7be3a546aa644281a0d3594395bbdf33`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:16.544429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d16f979d-1046-461f-aed3-1bd2c89bd339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:16.550609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 743/1474] 10.222.0.62 () {58 vars in 1059 bytes} [Fri May 27 03:18:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:16.558069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e5d42-eb1c-491f-98c0-bb377d0a3d6d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:16.568310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e5d42-eb1c-491f-98c0-bb377d0a3d6d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:16.571771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df026b8d-f433-473d-91a8-747a50b2bd58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7be3a546aa644281a0d3594395bbdf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:16.572413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df026b8d-f433-473d-91a8-747a50b2bd58 service glance] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:16.572698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df026b8d-f433-473d-91a8-747a50b2bd58 service glance] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:16.572975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df026b8d-f433-473d-91a8-747a50b2bd58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:16.581604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df026b8d-f433-473d-91a8-747a50b2bd58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:16.583019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df026b8d-f433-473d-91a8-747a50b2bd58 service glance] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:16.592883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 732/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:18:18.851070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de906-7f6b-4d24-b45a-6c66790cd3c6 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:18.851655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de906-7f6b-4d24-b45a-6c66790cd3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:18.851655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de906-7f6b-4d24-b45a-6c66790cd3c6 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:18.853027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 744/1476] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:18.858158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e1f26-8019-4327-88ea-406f386d1ad1 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:18.858551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e1f26-8019-4327-88ea-406f386d1ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:18.858717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e1f26-8019-4327-88ea-406f386d1ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:18.876559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-521e1f26-8019-4327-88ea-406f386d1ad1 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:18.877737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-521e1f26-8019-4327-88ea-406f386d1ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:18.886009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 733/1477] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:18.907690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93de906-7f6b-4d24-b45a-6c66790cd3c6 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:18.919892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93de906-7f6b-4d24-b45a-6c66790cd3c6 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:18.924251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9174b6f-4dd3-448b-9f9c-4aa317e1e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:18.924890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9174b6f-4dd3-448b-9f9c-4aa317e1e596 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:18.925172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9174b6f-4dd3-448b-9f9c-4aa317e1e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:18.925446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9174b6f-4dd3-448b-9f9c-4aa317e1e596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:18.926179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9174b6f-4dd3-448b-9f9c-4aa317e1e596 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:18.927949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-b9174b6f-4dd3-448b-9f9c-4aa317e1e596 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:18:18.927949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:18:18.927949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:18:18.929605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9174b6f-4dd3-448b-9f9c-4aa317e1e596 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:18.949075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 745/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Fri May 27 03:18:18 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:18:21.299412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e161-8767-42a4-a0ad-5418931b9542 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:21.299971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e161-8767-42a4-a0ad-5418931b9542 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:21.299971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e161-8767-42a4-a0ad-5418931b9542 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:21.301293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 734/1479] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:21.307043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5503b-5313-4546-8b07-7d406a9e6251 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:21.307489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5503b-5313-4546-8b07-7d406a9e6251 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:21.307626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5503b-5313-4546-8b07-7d406a9e6251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:21.325878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ffa5503b-5313-4546-8b07-7d406a9e6251 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:21.327071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ffa5503b-5313-4546-8b07-7d406a9e6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:21.334739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 746/1480] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:21.378017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9e161-8767-42a4-a0ad-5418931b9542 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:21.392611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9e161-8767-42a4-a0ad-5418931b9542 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:21.398040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7aea5-315a-417f-bd56-43e61430f8bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:21.398672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7aea5-315a-417f-bd56-43e61430f8bb service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:21.398958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7aea5-315a-417f-bd56-43e61430f8bb service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:21.399238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7aea5-315a-417f-bd56-43e61430f8bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:21.407177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7aea5-315a-417f-bd56-43e61430f8bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:21.408783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7aea5-315a-417f-bd56-43e61430f8bb service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:21.418105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 735/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:18:23.796763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42d534-9036-435f-a759-7b2e3776d05a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:23.797316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42d534-9036-435f-a759-7b2e3776d05a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:23.797316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42d534-9036-435f-a759-7b2e3776d05a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:23.798666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 747/1482] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:23.803912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68994b-5a0f-4f4e-958d-3f5cca767e30 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:23.804955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68994b-5a0f-4f4e-958d-3f5cca767e30 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:23.805084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68994b-5a0f-4f4e-958d-3f5cca767e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:23.822923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-bb68994b-5a0f-4f4e-958d-3f5cca767e30 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:23.824138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-bb68994b-5a0f-4f4e-958d-3f5cca767e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:23.831928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 736/1483] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:23.868162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42d534-9036-435f-a759-7b2e3776d05a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:23.878679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42d534-9036-435f-a759-7b2e3776d05a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:23.882191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e054ad6-8ec7-4ce2-b0fe-31e7ac16d696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:23.882821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054ad6-8ec7-4ce2-b0fe-31e7ac16d696 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:23.883109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054ad6-8ec7-4ce2-b0fe-31e7ac16d696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:23.883386 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054ad6-8ec7-4ce2-b0fe-31e7ac16d696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:23.892024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e054ad6-8ec7-4ce2-b0fe-31e7ac16d696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:23.893454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e054ad6-8ec7-4ce2-b0fe-31e7ac16d696 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:23.902877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 748/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:18:26.142513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b6586-867a-43e1-8d7a-16a68638c99f None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:26.143079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b6586-867a-43e1-8d7a-16a68638c99f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:26.143079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b6586-867a-43e1-8d7a-16a68638c99f None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:26.144478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 737/1485] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:26.149743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd778-2c67-4b6a-9e14-09f586cc8956 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:26.150209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd778-2c67-4b6a-9e14-09f586cc8956 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:26.150318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd778-2c67-4b6a-9e14-09f586cc8956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:26.168237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-bdfbd778-2c67-4b6a-9e14-09f586cc8956 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:26.169418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bdfbd778-2c67-4b6a-9e14-09f586cc8956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:26.177116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 749/1486] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:26.214529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b6586-867a-43e1-8d7a-16a68638c99f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:26.225024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b6586-867a-43e1-8d7a-16a68638c99f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:26.229015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0d738-76b4-47be-8433-8512ebfe3776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:26.229638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0d738-76b4-47be-8433-8512ebfe3776 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:26.229924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0d738-76b4-47be-8433-8512ebfe3776 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:26.230230 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0d738-76b4-47be-8433-8512ebfe3776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:26.238900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0d738-76b4-47be-8433-8512ebfe3776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:26.240367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0d738-76b4-47be-8433-8512ebfe3776 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:26.250219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 738/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:18:28.521418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb3bc-22a3-42b9-8648-1fba6e768f8c None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:28.521995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb3bc-22a3-42b9-8648-1fba6e768f8c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:28.521995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb3bc-22a3-42b9-8648-1fba6e768f8c None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:28.523340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 750/1488] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:28.528833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651d50-fb03-4b75-a045-fcf2b59f56c2 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:28.529279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651d50-fb03-4b75-a045-fcf2b59f56c2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:28.529372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651d50-fb03-4b75-a045-fcf2b59f56c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:28.547437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f4651d50-fb03-4b75-a045-fcf2b59f56c2 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:28.548631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f4651d50-fb03-4b75-a045-fcf2b59f56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:28.556832 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 739/1489] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:28.597280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decb3bc-22a3-42b9-8648-1fba6e768f8c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:28.608349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decb3bc-22a3-42b9-8648-1fba6e768f8c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:28.611750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674efc2d-23de-4b57-b03b-57db22735390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:28.612422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674efc2d-23de-4b57-b03b-57db22735390 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:28.612704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674efc2d-23de-4b57-b03b-57db22735390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:28.612975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674efc2d-23de-4b57-b03b-57db22735390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:28.625428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674efc2d-23de-4b57-b03b-57db22735390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:28.626939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674efc2d-23de-4b57-b03b-57db22735390 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:28.639434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 751/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:18:30.999485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0a15-99c9-4bcc-8ad5-cfbd38a64286 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:31.000058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0a15-99c9-4bcc-8ad5-cfbd38a64286 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:31.000058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0a15-99c9-4bcc-8ad5-cfbd38a64286 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:31.001456 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 740/1491] 10.222.0.62 () {54 vars in 926 bytes} [Fri May 27 03:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:31.006750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bceb49-2b43-45f8-a76e-77db44cec324 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:31.007801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bceb49-2b43-45f8-a76e-77db44cec324 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:31.007932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bceb49-2b43-45f8-a76e-77db44cec324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:31.026288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-00bceb49-2b43-45f8-a76e-77db44cec324 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:31.027481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-00bceb49-2b43-45f8-a76e-77db44cec324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:31.035211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 752/1492] 10.222.0.62 () {58 vars in 1023 bytes} [Fri May 27 03:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:18:31.074462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0a15-99c9-4bcc-8ad5-cfbd38a64286 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:31.085047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0a15-99c9-4bcc-8ad5-cfbd38a64286 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:31.088866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f18d1d-8baa-4917-bb4f-0136fa3a9265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:31.089496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18d1d-8baa-4917-bb4f-0136fa3a9265 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:31.089779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18d1d-8baa-4917-bb4f-0136fa3a9265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:31.090053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18d1d-8baa-4917-bb4f-0136fa3a9265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:31.098750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f18d1d-8baa-4917-bb4f-0136fa3a9265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:31.100227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f18d1d-8baa-4917-bb4f-0136fa3a9265 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:31.109368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 741/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:18:41.020163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=33 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:41.105192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=33 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:20:09.909746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ffed4-82e7-429d-b9c0-9dd5a6f53d0e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:09.910352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ffed4-82e7-429d-b9c0-9dd5a6f53d0e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:09.910352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ffed4-82e7-429d-b9c0-9dd5a6f53d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:09.928652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3f7ffed4-82e7-429d-b9c0-9dd5a6f53d0e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:20:09.930163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3f7ffed4-82e7-429d-b9c0-9dd5a6f53d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:20:09.937989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 753/1494] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:20:09.948114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031901f-c9ef-4b8d-8137-c1c1295db05d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:09.948617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031901f-c9ef-4b8d-8137-c1c1295db05d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:09.948716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031901f-c9ef-4b8d-8137-c1c1295db05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:09.966985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4031901f-c9ef-4b8d-8137-c1c1295db05d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:20:09.968198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4031901f-c9ef-4b8d-8137-c1c1295db05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:20:09.975911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 742/1495] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:20:09.980286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ffed4-82e7-429d-b9c0-9dd5a6f53d0e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:09.990107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ffed4-82e7-429d-b9c0-9dd5a6f53d0e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:09.995273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:09.995949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:09.996262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:09.996537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:09.997267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:20:09.997575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:09.998584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:09.998584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:20:09.998584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:20:09.999454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.011376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 754/1496] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:20:10.019239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031901f-c9ef-4b8d-8137-c1c1295db05d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.028170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031901f-c9ef-4b8d-8137-c1c1295db05d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.032673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.033319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.033606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.033882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.034578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.036372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING py.warnings [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:10.036372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: warnings.warn(msg) May 27 03:20:10.036372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: May 27 03:20:10.037560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.072112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 743/1497] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:20:10 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:20:10.076195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.085727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d08b5b-4979-4fc6-8e08-22b99c3721b1 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.140482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.141125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.141408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.141711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.142405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.143400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:10.143400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:20:10.143400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:20:10.144633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.162821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 755/1498] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:20:10 2022] POST /identity/v3/users => generated 289 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:20:10.166841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.177327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2461d-7df4-4fba-9dc7-a275b709b8cf admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.181843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.182495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.182781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.183057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.183841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.184865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING py.warnings [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:10.184865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: warnings.warn(msg) May 27 03:20:10.184865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: May 27 03:20:10.185739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.198665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 744/1499] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:20:10.202582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.212804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad881ed5-7e7d-42dd-9253-bcc10b882d92 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.217205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8297d7e-1fec-4916-acce-43eeaf4be4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.217967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8297d7e-1fec-4916-acce-43eeaf4be4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.218154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8297d7e-1fec-4916-acce-43eeaf4be4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.218519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8297d7e-1fec-4916-acce-43eeaf4be4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.219356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8297d7e-1fec-4916-acce-43eeaf4be4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.220391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-e8297d7e-1fec-4916-acce-43eeaf4be4f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:10.220391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:20:10.220391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:20:10.221266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8297d7e-1fec-4916-acce-43eeaf4be4f4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.234822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 756/1500] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:20:10.238796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.248343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9315-3cba-4dd6-9f7c-6770bea1db96 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.252829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b655c38-d48a-4c29-9f69-436a2ab42a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.253721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b655c38-d48a-4c29-9f69-436a2ab42a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.254006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b655c38-d48a-4c29-9f69-436a2ab42a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.254292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b655c38-d48a-4c29-9f69-436a2ab42a9f admin admin] PATH_INFO: `/v3/projects/b72e8f48ea4740c9a0f8efc4afea20c5/users/7b4b69d3fb42495b85601f299200015c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.269050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b655c38-d48a-4c29-9f69-436a2ab42a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72e8f48ea4740c9a0f8efc4afea20c5, user_id=7b4b69d3fb42495b85601f299200015c, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.270093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING py.warnings [None req-0b655c38-d48a-4c29-9f69-436a2ab42a9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:10.270093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: warnings.warn(msg) May 27 03:20:10.270093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: May 27 03:20:10.272629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b655c38-d48a-4c29-9f69-436a2ab42a9f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.286138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 745/1501] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:20:10 2022] PUT /identity/v3/projects/b72e8f48ea4740c9a0f8efc4afea20c5/users/7b4b69d3fb42495b85601f299200015c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:20:10.297254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bcdb2-5521-490b-8afc-f068cb28cebd None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.297552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bcdb2-5521-490b-8afc-f068cb28cebd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.297865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bcdb2-5521-490b-8afc-f068cb28cebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.323910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-708bcdb2-5521-490b-8afc-f068cb28cebd None None] MFA Rules not processed for user `7b4b69d3fb42495b85601f299200015c`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:20:10.325114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-708bcdb2-5521-490b-8afc-f068cb28cebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:20:10.388272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 757/1502] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:20:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:20:10.401111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc72013-3185-4fc9-9b33-ce7bd1435dce None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.401403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc72013-3185-4fc9-9b33-ce7bd1435dce None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.401685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc72013-3185-4fc9-9b33-ce7bd1435dce None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.403078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 746/1503] 10.222.0.62 () {54 vars in 963 bytes} [Fri May 27 03:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:20:10.408005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321df38-8029-4234-9b85-5fb218d69d20 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.408301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321df38-8029-4234-9b85-5fb218d69d20 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.408585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321df38-8029-4234-9b85-5fb218d69d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.427282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9321df38-8029-4234-9b85-5fb218d69d20 None None] MFA Rules not processed for user `4a8b4111f8774b03815a525fdbdaa4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:20:10.428472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9321df38-8029-4234-9b85-5fb218d69d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:20:10.470139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:20:10.478822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 758/1504] 10.222.0.62 () {58 vars in 1060 bytes} [Fri May 27 03:20:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:20:10.486303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc72013-3185-4fc9-9b33-ce7bd1435dce None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.497267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc72013-3185-4fc9-9b33-ce7bd1435dce None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.500775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.501402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.501690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.501972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.510006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.511428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.520095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 747/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:20:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:20:10.561416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64448-94c9-42d2-87f4-a24ea19003a9 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.561743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64448-94c9-42d2-87f4-a24ea19003a9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.562029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64448-94c9-42d2-87f4-a24ea19003a9 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.563351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 759/1506] 10.222.0.62 () {52 vars in 829 bytes} [Fri May 27 03:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:20:10.567052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.628002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a7d52-5e7d-4fcf-91d2-753d9bfc5e46 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.632531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.633430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.633719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.634002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] PATH_INFO: `/v3/users/7b4b69d3fb42495b85601f299200015c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.635184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4b69d3fb42495b85601f299200015c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.636239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING py.warnings [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:10.636239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: warnings.warn(msg) May 27 03:20:10.636239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: May 27 03:20:10.637435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.699278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4b69d3fb42495b85601f299200015c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:10.699991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4b69d3fb42495b85601f299200015c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:10.700622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4b69d3fb42495b85601f299200015c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:10.719340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4b69d3fb42495b85601f299200015c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:10.731888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4b69d3fb42495b85601f299200015c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:10.735814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:20:10.747106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 748/1507] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:20:10 2022] DELETE /identity/v3/users/7b4b69d3fb42495b85601f299200015c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:20:10.751490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.751783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.752085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.772516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:20:10.773909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:20:10.832475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 760/1508] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:20:10.842829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:10.844257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:20:10.900464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c38aee-9b7f-41b7-8b66-a42e8cea3b47 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:10.904040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882f354-be8d-469f-8991-63c7b6793505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:10.904683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882f354-be8d-469f-8991-63c7b6793505 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:10.904972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882f354-be8d-469f-8991-63c7b6793505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:10.905254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882f354-be8d-469f-8991-63c7b6793505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:10.915432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882f354-be8d-469f-8991-63c7b6793505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:10.917574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882f354-be8d-469f-8991-63c7b6793505 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:10.928270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 749/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:20:11.284932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:20:11.286481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:20:11.298174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df592977-8a59-44ba-b288-12adf9c3e28b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:20:11.302888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:20:11.303707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:20:11.304018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:20:11.304296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] PATH_INFO: `/v3/projects/b72e8f48ea4740c9a0f8efc4afea20c5` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:20:11.305528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72e8f48ea4740c9a0f8efc4afea20c5)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:20:11.308565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:20:11.308565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:20:11.308565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:20:11.308565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:20:11.370066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72e8f48ea4740c9a0f8efc4afea20c5', 'request_id': 'req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:11.384887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72e8f48ea4740c9a0f8efc4afea20c5', 'request_id': 'req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:11.390734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72e8f48ea4740c9a0f8efc4afea20c5', 'request_id': 'req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:11.405924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] The token cache is being invalidate because project b72e8f48ea4740c9a0f8efc4afea20c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:20:11.406198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:11.407020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db76c5f1-878a-4f3a-a654-8b70d0b9bb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:20:11.408497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 761/1510] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:20:11 2022] DELETE /identity/v3/projects/b72e8f48ea4740c9a0f8efc4afea20c5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:20:20.923957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:20:21.402815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:21:51.707466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cccdc3-e1bb-4052-a2b1-f7febd9564e1 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:51.708126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cccdc3-e1bb-4052-a2b1-f7febd9564e1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:51.708126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cccdc3-e1bb-4052-a2b1-f7febd9564e1 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:51.709427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:51.714548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710986c-b249-4041-91f6-ff56f3559c5c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:51.714974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710986c-b249-4041-91f6-ff56f3559c5c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:51.715111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710986c-b249-4041-91f6-ff56f3559c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:51.733284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7710986c-b249-4041-91f6-ff56f3559c5c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:51.734479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7710986c-b249-4041-91f6-ff56f3559c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:51.792607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:51.960924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cccdc3-e1bb-4052-a2b1-f7febd9564e1 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:51.962556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-26cccdc3-e1bb-4052-a2b1-f7febd9564e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:52.037873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cccdc3-e1bb-4052-a2b1-f7febd9564e1 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:52.043253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c85dff-4ebf-44a0-9dbf-a26f86600f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:52.043927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85dff-4ebf-44a0-9dbf-a26f86600f53 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:52.044216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85dff-4ebf-44a0-9dbf-a26f86600f53 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:52.044493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85dff-4ebf-44a0-9dbf-a26f86600f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:52.052518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85dff-4ebf-44a0-9dbf-a26f86600f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:52.054146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85dff-4ebf-44a0-9dbf-a26f86600f53 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:52.063398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 751/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:52.933144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7de1e3-efce-478a-a113-f6a2257d3b68 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:52.933674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7de1e3-efce-478a-a113-f6a2257d3b68 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:52.933674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7de1e3-efce-478a-a113-f6a2257d3b68 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:52.935003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:52.940006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a34d6-1a64-4f28-82e7-cb73a8990043 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:52.940418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a34d6-1a64-4f28-82e7-cb73a8990043 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:52.940565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a34d6-1a64-4f28-82e7-cb73a8990043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:52.958428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c80a34d6-1a64-4f28-82e7-cb73a8990043 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:52.959637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c80a34d6-1a64-4f28-82e7-cb73a8990043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:52.967686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:53.121345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7de1e3-efce-478a-a113-f6a2257d3b68 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:53.135545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7de1e3-efce-478a-a113-f6a2257d3b68 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:53.135714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a9db9-05fa-4d20-aa84-104c2633e48a None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:53.136522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a9db9-05fa-4d20-aa84-104c2633e48a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:53.136812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a9db9-05fa-4d20-aa84-104c2633e48a None None] PATH_INFO: `/v3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:53.138113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 753/1516] 10.222.0.60 () {54 vars in 925 bytes} [Fri May 27 03:21:53 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:21:53.141066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e9ccc-1b5c-40ca-aed5-38fbd920c72e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:53.141700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9ccc-1b5c-40ca-aed5-38fbd920c72e service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:53.141982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9ccc-1b5c-40ca-aed5-38fbd920c72e service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:53.142255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9ccc-1b5c-40ca-aed5-38fbd920c72e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:53.143928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2279d3-504f-4734-ad73-a848fa359382 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:53.144215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2279d3-504f-4734-ad73-a848fa359382 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:53.144493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2279d3-504f-4734-ad73-a848fa359382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:53.150559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e9ccc-1b5c-40ca-aed5-38fbd920c72e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:53.152233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e9ccc-1b5c-40ca-aed5-38fbd920c72e service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:53.162360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-3b2279d3-504f-4734-ad73-a848fa359382 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:53.162615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 764/1517] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:53.163620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3b2279d3-504f-4734-ad73-a848fa359382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:53.230076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 754/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Fri May 27 03:21:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:22:02.170193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bf2d-7217-4e7b-ab73-c3123e0fdde0 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:02.170721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bf2d-7217-4e7b-ab73-c3123e0fdde0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:02.170821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bf2d-7217-4e7b-ab73-c3123e0fdde0 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:02.172212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 765/1519] 10.222.0.62 () {54 vars in 960 bytes} [Fri May 27 03:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:22:02.177033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa01f8c-0d02-4149-b1b9-cc6605395c19 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:02.177462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa01f8c-0d02-4149-b1b9-cc6605395c19 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:02.177593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa01f8c-0d02-4149-b1b9-cc6605395c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:02.195608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-bfa01f8c-0d02-4149-b1b9-cc6605395c19 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:22:02.196852 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-bfa01f8c-0d02-4149-b1b9-cc6605395c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:02.205897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 755/1520] 10.222.0.62 () {58 vars in 1057 bytes} [Fri May 27 03:22:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:22:02.213323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8bf2d-7217-4e7b-ab73-c3123e0fdde0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:22:02.226269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8bf2d-7217-4e7b-ab73-c3123e0fdde0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:22:02.231122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92090e6e-b0fb-42ab-8d26-dac8a2465317 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:22:02.231743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92090e6e-b0fb-42ab-8d26-dac8a2465317 service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:02.232066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92090e6e-b0fb-42ab-8d26-dac8a2465317 service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:02.232348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92090e6e-b0fb-42ab-8d26-dac8a2465317 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:02.240397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92090e6e-b0fb-42ab-8d26-dac8a2465317 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:22:02.241967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92090e6e-b0fb-42ab-8d26-dac8a2465317 service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:22:02.251330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 766/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:22:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:22:04.446495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4856a2a-ee7f-450b-b012-e9307d077815 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:04.447100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4856a2a-ee7f-450b-b012-e9307d077815 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:04.447100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4856a2a-ee7f-450b-b012-e9307d077815 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:04.448462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:22:04.453161 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139285e3-c45e-472f-a056-7b690f3389bd None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:04.453616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139285e3-c45e-472f-a056-7b690f3389bd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:04.453727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139285e3-c45e-472f-a056-7b690f3389bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:04.472644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-139285e3-c45e-472f-a056-7b690f3389bd None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:22:04.473836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-139285e3-c45e-472f-a056-7b690f3389bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:04.481454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:22:04.633272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4856a2a-ee7f-450b-b012-e9307d077815 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:22:04.648016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4856a2a-ee7f-450b-b012-e9307d077815 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:22:04.653279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11fe7a-61bb-497b-9723-8f6bf87a92d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:22:04.653916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11fe7a-61bb-497b-9723-8f6bf87a92d3 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:04.654200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11fe7a-61bb-497b-9723-8f6bf87a92d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:04.654478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11fe7a-61bb-497b-9723-8f6bf87a92d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:04.662966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11fe7a-61bb-497b-9723-8f6bf87a92d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:22:04.664615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11fe7a-61bb-497b-9723-8f6bf87a92d3 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:22:04.674268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 757/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:22:05.534975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72311986-c446-4abb-818a-57de731f391a None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:05.535500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72311986-c446-4abb-818a-57de731f391a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:05.535620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72311986-c446-4abb-818a-57de731f391a None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:05.536947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:22:05.541888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba8be6-b60d-4e82-b056-ed434ecea7c2 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:05.542292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba8be6-b60d-4e82-b056-ed434ecea7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:05.542454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba8be6-b60d-4e82-b056-ed434ecea7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:05.560641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-81ba8be6-b60d-4e82-b056-ed434ecea7c2 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:22:05.561825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-81ba8be6-b60d-4e82-b056-ed434ecea7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:05.570354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:22:05.718712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72311986-c446-4abb-818a-57de731f391a None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:22:05.733117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72311986-c446-4abb-818a-57de731f391a None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:22:05.738990 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f68e9-3e5a-47f7-8585-8defa85c2de8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:22:05.739619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f68e9-3e5a-47f7-8585-8defa85c2de8 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:05.739936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f68e9-3e5a-47f7-8585-8defa85c2de8 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:05.740215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f68e9-3e5a-47f7-8585-8defa85c2de8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:05.748008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f68e9-3e5a-47f7-8585-8defa85c2de8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:22:05.749592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f68e9-3e5a-47f7-8585-8defa85c2de8 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:22:05.758789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 769/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:22:10.073039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29d63b-d09c-4783-b39a-8f6a12df24c1 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:10.073645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29d63b-d09c-4783-b39a-8f6a12df24c1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:10.073645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29d63b-d09c-4783-b39a-8f6a12df24c1 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:10.074995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:22:10.080699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8915c-9d58-4503-83d7-275be60beab3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:10.081125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8915c-9d58-4503-83d7-275be60beab3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:10.081251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8915c-9d58-4503-83d7-275be60beab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:10.099790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0ba8915c-9d58-4503-83d7-275be60beab3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:22:10.101001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0ba8915c-9d58-4503-83d7-275be60beab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:10.108659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:22:10.144551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29d63b-d09c-4783-b39a-8f6a12df24c1 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:22:10.146258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3c29d63b-d09c-4783-b39a-8f6a12df24c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:10.198570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:22:10.203071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29d63b-d09c-4783-b39a-8f6a12df24c1 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:22:10.207239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1568a-4ddd-4c9e-a98c-0c1c28eb70ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:22:10.207909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1568a-4ddd-4c9e-a98c-0c1c28eb70ee service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:10.208197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1568a-4ddd-4c9e-a98c-0c1c28eb70ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:10.208476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1568a-4ddd-4c9e-a98c-0c1c28eb70ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:10.217473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1568a-4ddd-4c9e-a98c-0c1c28eb70ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:22:10.218897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1568a-4ddd-4c9e-a98c-0c1c28eb70ee service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:22:10.228057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 760/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:22:12.467234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c32e9-e8ee-4fa6-a7c2-3678e37093b3 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:12.467810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c32e9-e8ee-4fa6-a7c2-3678e37093b3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:12.467810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c32e9-e8ee-4fa6-a7c2-3678e37093b3 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:12.469356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:22:12.474674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7153b4-c420-4728-836e-2801a0302a3e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:12.475141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7153b4-c420-4728-836e-2801a0302a3e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:12.475262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7153b4-c420-4728-836e-2801a0302a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:12.494740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fb7153b4-c420-4728-836e-2801a0302a3e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:22:12.495979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fb7153b4-c420-4728-836e-2801a0302a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:12.504210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:22:12.541243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c32e9-e8ee-4fa6-a7c2-3678e37093b3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:22:12.551767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c32e9-e8ee-4fa6-a7c2-3678e37093b3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:22:12.555238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e422cd-675f-46eb-b236-859238b9b8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:22:12.555898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e422cd-675f-46eb-b236-859238b9b8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:12.556187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e422cd-675f-46eb-b236-859238b9b8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:12.556465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e422cd-675f-46eb-b236-859238b9b8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:12.565595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e422cd-675f-46eb-b236-859238b9b8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:22:12.567024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e422cd-675f-46eb-b236-859238b9b8d0 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:22:12.576156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 772/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:22:16.175057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23861c22-5228-416e-8f67-05e64b6233b7 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:16.175631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23861c22-5228-416e-8f67-05e64b6233b7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:16.175631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23861c22-5228-416e-8f67-05e64b6233b7 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:16.177003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:22:16.182225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3257-204a-4d56-a2aa-3804dc054a06 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:16.182627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3257-204a-4d56-a2aa-3804dc054a06 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:16.182821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3257-204a-4d56-a2aa-3804dc054a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:16.200732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ddef3257-204a-4d56-a2aa-3804dc054a06 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:22:16.201874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ddef3257-204a-4d56-a2aa-3804dc054a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:16.210291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:22:16.478894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23861c22-5228-416e-8f67-05e64b6233b7 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:22:16.480594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-23861c22-5228-416e-8f67-05e64b6233b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:16.537586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23861c22-5228-416e-8f67-05e64b6233b7 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:22:16.541123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dd05d-17cd-4de4-a8a2-e28bba20772e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:22:16.541796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dd05d-17cd-4de4-a8a2-e28bba20772e service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:16.542083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dd05d-17cd-4de4-a8a2-e28bba20772e service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:16.542365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dd05d-17cd-4de4-a8a2-e28bba20772e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:16.551678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dd05d-17cd-4de4-a8a2-e28bba20772e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:22:16.553152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dd05d-17cd-4de4-a8a2-e28bba20772e service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:22:16.562351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 763/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:22:18.757302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf5fe-67ea-4340-9d1f-568baa0124dc None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:18.757893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf5fe-67ea-4340-9d1f-568baa0124dc None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:18.757893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf5fe-67ea-4340-9d1f-568baa0124dc None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:18.759217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:22:18.764460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0012cc-7975-4002-a8bc-6b833fb0bc8d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:18.764897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0012cc-7975-4002-a8bc-6b833fb0bc8d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:18.765015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0012cc-7975-4002-a8bc-6b833fb0bc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:18.783425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-cf0012cc-7975-4002-a8bc-6b833fb0bc8d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:22:18.784646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cf0012cc-7975-4002-a8bc-6b833fb0bc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:18.792376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:22:19.060549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cf5fe-67ea-4340-9d1f-568baa0124dc None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:22:19.062242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-119cf5fe-67ea-4340-9d1f-568baa0124dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:22:19.074973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cf5fe-67ea-4340-9d1f-568baa0124dc None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:22:19.079547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb691da-dfd2-4af7-bcf2-e0ab26dc3112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:22:19.080238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb691da-dfd2-4af7-bcf2-e0ab26dc3112 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:22:19.080526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb691da-dfd2-4af7-bcf2-e0ab26dc3112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:22:19.080808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb691da-dfd2-4af7-bcf2-e0ab26dc3112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:22:19.090170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb691da-dfd2-4af7-bcf2-e0ab26dc3112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:22:19.091583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb691da-dfd2-4af7-bcf2-e0ab26dc3112 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:22:19.101375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 775/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:22:28.777660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=36 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:22:29.097294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=66 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:47.466080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae8ba5-146a-482c-acf2-9cfa0b9477b4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.466723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae8ba5-146a-482c-acf2-9cfa0b9477b4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.466723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae8ba5-146a-482c-acf2-9cfa0b9477b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.495500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ccae8ba5-146a-482c-acf2-9cfa0b9477b4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.496848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ccae8ba5-146a-482c-acf2-9cfa0b9477b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.511195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 765/1540] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.522585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e80b6-d99b-4c4a-aba7-6412dde7d908 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.523868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e80b6-d99b-4c4a-aba7-6412dde7d908 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.524408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e80b6-d99b-4c4a-aba7-6412dde7d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.562968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-349e80b6-d99b-4c4a-aba7-6412dde7d908 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.564243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-349e80b6-d99b-4c4a-aba7-6412dde7d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.582147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 776/1541] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:47.587479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae8ba5-146a-482c-acf2-9cfa0b9477b4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.602922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae8ba5-146a-482c-acf2-9cfa0b9477b4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.608250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.608934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.609218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.609647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.610405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:47.610716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.611762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING py.warnings [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:47.611762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: warnings.warn(msg) May 27 03:23:47.611762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: May 27 03:23:47.617129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041d58-647a-4e1d-8f10-a775013f6a18 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.632190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 766/1542] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:47.640485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e80b6-d99b-4c4a-aba7-6412dde7d908 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.654353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e80b6-d99b-4c4a-aba7-6412dde7d908 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.656146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b346c8-c8d6-4755-af44-7742b446774a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.656441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b346c8-c8d6-4755-af44-7742b446774a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.657448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b346c8-c8d6-4755-af44-7742b446774a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.665641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.666323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.666607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.666904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.667703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.668890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:47.668890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:23:47.668890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:23:47.670672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.689161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c3b346c8-c8d6-4755-af44-7742b446774a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.690371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c3b346c8-c8d6-4755-af44-7742b446774a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.699097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 767/1543] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:47.700445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080562-b468-4d73-9d47-640f0eda686d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.701109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080562-b468-4d73-9d47-640f0eda686d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.701395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53080562-b468-4d73-9d47-640f0eda686d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.723513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-53080562-b468-4d73-9d47-640f0eda686d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.724668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-53080562-b468-4d73-9d47-640f0eda686d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.738086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 768/1544] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.739378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb506-8ad6-4a9b-9231-f8073682c8c4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.739663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb506-8ad6-4a9b-9231-f8073682c8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.739960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb506-8ad6-4a9b-9231-f8073682c8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.742004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 777/1545] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:47.745943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.768971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-785cb506-8ad6-4a9b-9231-f8073682c8c4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.770192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-785cb506-8ad6-4a9b-9231-f8073682c8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.838167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6dbac-4be8-4985-978d-d99cb48a1923 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.843236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12152698-cc2a-462f-98c2-d33cdfb33e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.843939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152698-cc2a-462f-98c2-d33cdfb33e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.844275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152698-cc2a-462f-98c2-d33cdfb33e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.844557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152698-cc2a-462f-98c2-d33cdfb33e69 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.845269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12152698-cc2a-462f-98c2-d33cdfb33e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.847258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12152698-cc2a-462f-98c2-d33cdfb33e69 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.857948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 769/1546] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.859320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792fbb5-5502-4774-b404-e7698dfaee65 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.859615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792fbb5-5502-4774-b404-e7698dfaee65 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.859915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792fbb5-5502-4774-b404-e7698dfaee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.873978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 778/1547] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:47.875284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db9b46-e3f3-48c0-b3fa-35135b2f7e23 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.875637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db9b46-e3f3-48c0-b3fa-35135b2f7e23 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.875967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db9b46-e3f3-48c0-b3fa-35135b2f7e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.883319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d792fbb5-5502-4774-b404-e7698dfaee65 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.884788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d792fbb5-5502-4774-b404-e7698dfaee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.892759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 770/1548] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.893892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38dd66-b7ac-47a4-9770-4bc6e4c8b69a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.894103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-48db9b46-e3f3-48c0-b3fa-35135b2f7e23 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.894212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38dd66-b7ac-47a4-9770-4bc6e4c8b69a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.894441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38dd66-b7ac-47a4-9770-4bc6e4c8b69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.895287 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-48db9b46-e3f3-48c0-b3fa-35135b2f7e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.903929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 779/1549] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.905618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2055a83-5543-4856-8440-30d8ed3c5be5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.905618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2055a83-5543-4856-8440-30d8ed3c5be5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.905618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2055a83-5543-4856-8440-30d8ed3c5be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.922299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2c38dd66-b7ac-47a4-9770-4bc6e4c8b69a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.923506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2c38dd66-b7ac-47a4-9770-4bc6e4c8b69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.924298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-e2055a83-5543-4856-8440-30d8ed3c5be5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.925471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e2055a83-5543-4856-8440-30d8ed3c5be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.932164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 771/1550] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.933503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058117c4-875b-41ae-ae98-6ec0f486cc20 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.933834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058117c4-875b-41ae-ae98-6ec0f486cc20 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.934121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058117c4-875b-41ae-ae98-6ec0f486cc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.936105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 780/1551] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.944120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2055a83-5543-4856-8440-30d8ed3c5be5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.953778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2055a83-5543-4856-8440-30d8ed3c5be5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.958494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.959143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.959426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.959701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.960483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:47.960803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.961286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-058117c4-875b-41ae-ae98-6ec0f486cc20 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.962394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.962497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-058117c4-875b-41ae-ae98-6ec0f486cc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.970239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 772/1552] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.971016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058117c4-875b-41ae-ae98-6ec0f486cc20 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.986061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 781/1553] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:47.986061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.986366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058117c4-875b-41ae-ae98-6ec0f486cc20 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.989325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8473c-0d61-44de-8ab2-dce79cbbdabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.989997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8473c-0d61-44de-8ab2-dce79cbbdabf admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.990283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8473c-0d61-44de-8ab2-dce79cbbdabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.990562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8473c-0d61-44de-8ab2-dce79cbbdabf admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.991301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8473c-0d61-44de-8ab2-dce79cbbdabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.991505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3bdea-900c-4a88-b153-bc6bf80f6a04 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.992957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8473c-0d61-44de-8ab2-dce79cbbdabf admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.996500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e1afd-8454-4d1d-9a99-06c194d1feac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.997270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e1afd-8454-4d1d-9a99-06c194d1feac admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.998078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e1afd-8454-4d1d-9a99-06c194d1feac admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.998364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e1afd-8454-4d1d-9a99-06c194d1feac admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.999098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e1afd-8454-4d1d-9a99-06c194d1feac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:47.999411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e1afd-8454-4d1d-9a99-06c194d1feac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.001046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e1afd-8454-4d1d-9a99-06c194d1feac admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.006719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 773/1554] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:48.006719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe9a35-751a-46fc-85e1-7581214b3bfa None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.006719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe9a35-751a-46fc-85e1-7581214b3bfa None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.006719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe9a35-751a-46fc-85e1-7581214b3bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.013499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 782/1555] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:48.014334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108e8d-627c-450a-bb23-c456c788d160 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.014633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108e8d-627c-450a-bb23-c456c788d160 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.014918 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108e8d-627c-450a-bb23-c456c788d160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.024126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8bbe9a35-751a-46fc-85e1-7581214b3bfa None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:48.025348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8bbe9a35-751a-46fc-85e1-7581214b3bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.032486 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-23108e8d-627c-450a-bb23-c456c788d160 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:48.033648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-23108e8d-627c-450a-bb23-c456c788d160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.033777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 774/1556] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:48.034916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991bdae-f445-4868-bc12-983a061c6382 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.035207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991bdae-f445-4868-bc12-983a061c6382 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.035481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991bdae-f445-4868-bc12-983a061c6382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.043157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 783/1557] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:48.044822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82442a70-9873-4499-ab23-2eba853b120f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.045115 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82442a70-9873-4499-ab23-2eba853b120f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.045396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82442a70-9873-4499-ab23-2eba853b120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.057940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-6991bdae-f445-4868-bc12-983a061c6382 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:48.059117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6991bdae-f445-4868-bc12-983a061c6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.062681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-82442a70-9873-4499-ab23-2eba853b120f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:48.063849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-82442a70-9873-4499-ab23-2eba853b120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.066828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 775/1558] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:48.068054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e99c5b-e9a6-40b3-b3dc-fca133fa273d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.068336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e99c5b-e9a6-40b3-b3dc-fca133fa273d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.068615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e99c5b-e9a6-40b3-b3dc-fca133fa273d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.072558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 784/1559] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:48.076637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82442a70-9873-4499-ab23-2eba853b120f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.086057 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82442a70-9873-4499-ab23-2eba853b120f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.087271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-19e99c5b-e9a6-40b3-b3dc-fca133fa273d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:48.088505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-19e99c5b-e9a6-40b3-b3dc-fca133fa273d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.095528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.096219 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.096646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.096965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.097624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.099550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.101421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 776/1560] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:48.102292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e99c5b-e9a6-40b3-b3dc-fca133fa273d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.112390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e99c5b-e9a6-40b3-b3dc-fca133fa273d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.117047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.117701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.117989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.118338 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.119012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.120959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.129163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 785/1561] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.130166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.131345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 777/1562] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:48.135120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.144711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58911-dfef-4363-854d-22ec42ba5248 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.183245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:48.199628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86812f-5a25-40f0-98b7-87d8462a6062 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.204661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71b08e-2c1d-48c0-ad21-1830c2ad1657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.205464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71b08e-2c1d-48c0-ad21-1830c2ad1657 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.205693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.205824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71b08e-2c1d-48c0-ad21-1830c2ad1657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.206017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71b08e-2c1d-48c0-ad21-1830c2ad1657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.206354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.206640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.206736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71b08e-2c1d-48c0-ad21-1830c2ad1657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.206921 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.207980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:48.208294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.208987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71b08e-2c1d-48c0-ad21-1830c2ad1657 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.209907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.219072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:48.223298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 778/1563] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:48.224509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.235352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0f2e-d8ba-4813-ab4b-eeb0c154fa71 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.240447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 786/1564] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.240447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca438-8119-4aa0-9813-bb9ee11d8658 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.240447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca438-8119-4aa0-9813-bb9ee11d8658 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.240447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca438-8119-4aa0-9813-bb9ee11d8658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.256575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0efca438-8119-4aa0-9813-bb9ee11d8658 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:48.257807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0efca438-8119-4aa0-9813-bb9ee11d8658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.290604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.291260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.291561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.291812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.292555 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:48.292856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.295080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.306198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 779/1565] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:48.306979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.321965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 787/1566] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:48.322330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688f199-6f7a-4670-a1ed-a9d1241ea235 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.322804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca438-8119-4aa0-9813-bb9ee11d8658 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.327298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.328000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.328292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.328570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.329300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:48.329607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.331187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.331908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca438-8119-4aa0-9813-bb9ee11d8658 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.336696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.337333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.337621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.337905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.338667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:48.338980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.340601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.345589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 780/1567] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:48.346347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.357539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 788/1568] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:48.357539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.360250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24652f-cfe6-4a57-942d-372173563c87 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.364928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.365100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf7289-5ce5-4376-a246-d2594a7940c4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.365599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.365880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.366152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.366842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.367897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING py.warnings [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:48.367897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: warnings.warn(msg) May 27 03:23:48.367897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: May 27 03:23:48.369082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.373567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.374459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.374877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.375163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] PATH_INFO: `/v3/projects/3931210010614a1f987661568f29e851/users/720bd38555404aafa93db5ae49b13acf/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.389747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 781/1569] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.390562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.392654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3931210010614a1f987661568f29e851, user_id=720bd38555404aafa93db5ae49b13acf, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.393810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:48.393810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:23:48.393810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:23:48.395766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.400006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12191c3e-de08-4946-9c70-f9562e6d18d8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.405707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.406351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.406635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.406909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.407599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.409623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 789/1570] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:48 2022] PUT /identity/v3/projects/3931210010614a1f987661568f29e851/users/720bd38555404aafa93db5ae49b13acf/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:48.413345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.413478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.439931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 782/1571] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.439931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.473960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd43bb-a23d-4e19-9211-c5d5637ad985 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.498182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe849e-07f1-4811-b9fb-5708d3b31986 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.503141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.503795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.504114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.504395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.505114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.507042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.514585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:48.534068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 783/1572] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.534331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.534890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.535102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.535378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.535868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.536100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.538019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.561772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 790/1573] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.562592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.571804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f21e-c98f-4f4c-a490-18c4f712da9a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.572102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:48.605855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec5356-db95-4117-8486-3c6af3948f99 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.610807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.611476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.612203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.612490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.613237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:48.613909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.616045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.627709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 784/1574] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:48.628519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.634852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.635562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.635879 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.636166 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.636894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.639431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.643840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f32e5c-b7da-4a8c-b9f7-64f5de4fe585 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.648404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc73a1-bf34-4abb-bc3d-ae35e5867c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.649039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc73a1-bf34-4abb-bc3d-ae35e5867c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.649320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc73a1-bf34-4abb-bc3d-ae35e5867c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.649597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc73a1-bf34-4abb-bc3d-ae35e5867c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.650279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc73a1-bf34-4abb-bc3d-ae35e5867c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.652192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc73a1-bf34-4abb-bc3d-ae35e5867c71 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.671031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 791/1575] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.671031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.682401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 785/1576] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.682401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22602a44-c4fe-4d3f-b9ec-1f4b079c8f66 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.682401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22602a44-c4fe-4d3f-b9ec-1f4b079c8f66 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.682401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22602a44-c4fe-4d3f-b9ec-1f4b079c8f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.697794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-22602a44-c4fe-4d3f-b9ec-1f4b079c8f66 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:48.699280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-22602a44-c4fe-4d3f-b9ec-1f4b079c8f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.734231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745ca8a-32c4-4e51-9de1-744c26ae8fe1 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.758307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 786/1577] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:48.758953 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22602a44-c4fe-4d3f-b9ec-1f4b079c8f66 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.769674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22602a44-c4fe-4d3f-b9ec-1f4b079c8f66 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.774225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.775177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.775177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.775428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.776165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.778085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.793549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.794275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.794552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.794831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.795563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.797192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.799294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 787/1578] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.800131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.802813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:48.811288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 792/1579] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:48.811288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.811584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f428c-c68b-49a7-95b0-bd890f464315 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.815562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.816260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.816985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.816985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.817506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.819428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.820165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343add3-3c18-4f8d-bab9-a972fccb2bec admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.824670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.825951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.826239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.826519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.827403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.829408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.839996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 793/1580] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:48.840778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.847204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 788/1581] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.847204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.852477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3bded-4d33-45d5-b62a-e66e948158ad admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.854846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd143861-d740-43cd-83de-d036565b875a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.857075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.857711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.858000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.858278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.858964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.859613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.860289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.860603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.860907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.861021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.861587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.863510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.885056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 789/1582] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/users => generated 291 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.885931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.890427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 794/1583] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.891161 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.898635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4ff7e-d2ea-4bf2-969f-8882c02dcc7c admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.944843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f424f24-6193-4d79-93ef-9a1251553174 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.948392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.949038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.949328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.949608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.957595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:48.971873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.972554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.972843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.973122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.973822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.975814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.999576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 790/1584] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:48 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:48.999576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.006831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61135a-e9e3-4fc1-a6de-7baec108c208 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.011463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.012080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.012373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.012729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.014489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.015840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.016806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.018300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.026432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 791/1585] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.027279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.030370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 795/1586] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:49.031199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.039860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d66cf3-ae79-432c-b518-c807148c69da admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.040481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d6e3-9a91-426e-9e52-21b003fb69c0 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.045091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.045807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.045953 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.046257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.046476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.046724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.046870 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.047596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.047738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.048435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.049579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.050037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.064316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 796/1587] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.064510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 792/1588] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.064612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.064717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.073020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802b14b-a540-4ff1-9bc1-294936aecbe7 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.076870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d258cd-ecf6-4d20-8d70-09609e73007d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.078773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.079401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.079724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.080001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.080764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.082623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.083198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.084010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.084435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.084779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.085720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.087270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.096717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 797/1589] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.096717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.101193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 793/1590] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.102397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.106629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5411907-7789-44c9-8f51-e54ea9075b80 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.111705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.112443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.112729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.113007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.113123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44314b-b054-4207-a4bd-17030760b098 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.113763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.116181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.118507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.119374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.119734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.120114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.121006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.123345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.138549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 794/1591] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.138549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.141763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 798/1592] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:49.142666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.152747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190abe5b-24c3-4691-886d-dcec4bd6478e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.155167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39be5e-ccb0-46ee-800f-eebffaa8100c admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.158242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.159267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.159656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.160030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] PATH_INFO: `/v3/projects/5e539cb7b0fe47d6bdece2805d4ad0c1/users/e8fddb1cf74d4135b41000f8a6293020/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.160614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.161455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.161851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.162253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.164002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.165103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.177803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e539cb7b0fe47d6bdece2805d4ad0c1, user_id=e8fddb1cf74d4135b41000f8a6293020, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.178874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 795/1593] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.179801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.181058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.191035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a41ff0-496b-42e1-8799-48d41b4060f2 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.196757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 799/1594] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:49 2022] PUT /identity/v3/projects/5e539cb7b0fe47d6bdece2805d4ad0c1/users/e8fddb1cf74d4135b41000f8a6293020/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.197659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.215539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:49.265553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.267382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.267382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.267757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] PATH_INFO: `/v3/projects/b114bd896cc6469184097ef9300613da/users/b34132c9382a44cd97bc2c4e40c97eec/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.275555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a1fab-e6c4-4a22-8f6f-727684e2999a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.281386 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.282133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.282565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.283031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.283981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.285717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.286935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b114bd896cc6469184097ef9300613da, user_id=b34132c9382a44cd97bc2c4e40c97eec, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.291621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.297931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 800/1595] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.298873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.308797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 796/1596] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:49 2022] PUT /identity/v3/projects/b114bd896cc6469184097ef9300613da/users/b34132c9382a44cd97bc2c4e40c97eec/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.311066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.312954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370f01a-c546-4e4d-aaff-6fbecfb6333c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.376828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.377533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.377816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.378122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.378861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.380527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.383122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d199c2-e053-4c8f-b920-831663b9def5 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.389065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.389817 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.390115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.390464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.391380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.393206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.395001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 801/1597] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.396321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.398908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:49.404849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 797/1598] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.405758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.411221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461dc7-90f3-4a8e-bf00-55de48fb9e65 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.417302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b95422-ab23-4a95-b34f-3322d591788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.417811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1dd98-8c6e-4db3-bcd8-8112cb0dd119 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.418052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b95422-ab23-4a95-b34f-3322d591788e admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.418346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b95422-ab23-4a95-b34f-3322d591788e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.418684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b95422-ab23-4a95-b34f-3322d591788e admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.420070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b95422-ab23-4a95-b34f-3322d591788e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.421617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b95422-ab23-4a95-b34f-3322d591788e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.423579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf9233-2ede-4aa4-a94e-0ff7f3ec7909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.424617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf9233-2ede-4aa4-a94e-0ff7f3ec7909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.425008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf9233-2ede-4aa4-a94e-0ff7f3ec7909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.425389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf9233-2ede-4aa4-a94e-0ff7f3ec7909 admin admin] PATH_INFO: `/v3/projects/de1626e5c02845f5b9b93e8b5fe916f0/users/e4d835d78d4f4b019ced356585af16c7/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.432282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 802/1599] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.433446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde80d-db6e-4440-915b-3505574c136f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.433791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde80d-db6e-4440-915b-3505574c136f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.434082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde80d-db6e-4440-915b-3505574c136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.450012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf9233-2ede-4aa4-a94e-0ff7f3ec7909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1626e5c02845f5b9b93e8b5fe916f0, user_id=e4d835d78d4f4b019ced356585af16c7, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.452952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf9233-2ede-4aa4-a94e-0ff7f3ec7909 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.456095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0bdde80d-db6e-4440-915b-3505574c136f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:49.457765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0bdde80d-db6e-4440-915b-3505574c136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:49.468409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 803/1600] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:49.471557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdde80d-db6e-4440-915b-3505574c136f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.473083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 798/1601] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:49 2022] PUT /identity/v3/projects/de1626e5c02845f5b9b93e8b5fe916f0/users/e4d835d78d4f4b019ced356585af16c7/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.474699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f79b2-0359-430a-97ae-1c5ac35ba418 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.475342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f79b2-0359-430a-97ae-1c5ac35ba418 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.475962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f79b2-0359-430a-97ae-1c5ac35ba418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.482846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdde80d-db6e-4440-915b-3505574c136f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.495413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ec1f79b2-0359-430a-97ae-1c5ac35ba418 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:49.496692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ec1f79b2-0359-430a-97ae-1c5ac35ba418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:49.563600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.565557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.566192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.566787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] PATH_INFO: `/v3/projects/ac9d7226b54c494893714280b2a9be32/users/9c5ba513cf6b43c7b4908d6ef75ee493/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.572396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 799/1602] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:49.573234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f79b2-0359-430a-97ae-1c5ac35ba418 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.583059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f79b2-0359-430a-97ae-1c5ac35ba418 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.588453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.588608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9d7226b54c494893714280b2a9be32, user_id=9c5ba513cf6b43c7b4908d6ef75ee493, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.589446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.589781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.590135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] PATH_INFO: `/v3/projects/7ba6e3acda8d4e41af12c7a79bf3bc3b/users/f79a95f2221146a280a9d338c30be98a/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.591689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.608272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 804/1603] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:49 2022] PUT /identity/v3/projects/ac9d7226b54c494893714280b2a9be32/users/9c5ba513cf6b43c7b4908d6ef75ee493/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.609058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.612384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba6e3acda8d4e41af12c7a79bf3bc3b, user_id=f79a95f2221146a280a9d338c30be98a, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.615330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.627305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:49.635967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 800/1604] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:49 2022] PUT /identity/v3/projects/7ba6e3acda8d4e41af12c7a79bf3bc3b/users/f79a95f2221146a280a9d338c30be98a/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.635967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.688047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf846-38bc-40ff-8e44-adb9447e4838 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.710686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c64fa-3b82-4ca9-85d9-a521601306f7 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.717261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e46c-b1ce-4056-99d4-921234037c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.718534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e46c-b1ce-4056-99d4-921234037c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.718984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e46c-b1ce-4056-99d4-921234037c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.719790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e46c-b1ce-4056-99d4-921234037c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.720810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7e46c-b1ce-4056-99d4-921234037c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.723615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7e46c-b1ce-4056-99d4-921234037c02 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.738791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 801/1605] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:49.741057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.741352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.741663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.750046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.750960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.751240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.751525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] PATH_INFO: `/v3/projects/9943a29a4d00458983620f79e7e07765/users/0f04eeaae0a54cdea28b5a9760c1a864/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.761247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:49.762384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:49.766483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9943a29a4d00458983620f79e7e07765, user_id=0f04eeaae0a54cdea28b5a9760c1a864, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.769249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.774868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 802/1606] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:49.774868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.776809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:49.789524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 805/1607] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:49 2022] PUT /identity/v3/projects/9943a29a4d00458983620f79e7e07765/users/0f04eeaae0a54cdea28b5a9760c1a864/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.790654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.850192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:49.866872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8d00a-1129-4613-9584-f20f5c7abca2 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.867374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd177b5-1e12-4e7a-83cc-597bd45770ae None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.872315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442f0d5-5a20-4324-b839-42d90bc13570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.873021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442f0d5-5a20-4324-b839-42d90bc13570 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.873315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442f0d5-5a20-4324-b839-42d90bc13570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.873601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442f0d5-5a20-4324-b839-42d90bc13570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.876048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb90f16-a736-43d0-98df-1296ad70f7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.877052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb90f16-a736-43d0-98df-1296ad70f7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.877514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb90f16-a736-43d0-98df-1296ad70f7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.878267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb90f16-a736-43d0-98df-1296ad70f7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.941213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:49.954178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb90f16-a736-43d0-98df-1296ad70f7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.954505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442f0d5-5a20-4324-b839-42d90bc13570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.956264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442f0d5-5a20-4324-b839-42d90bc13570 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.956489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb90f16-a736-43d0-98df-1296ad70f7f8 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.968664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 803/1608] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:49.970044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 806/1609] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:49.970164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac55e-6cdb-4735-b3fb-a8314f169786 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.970923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac55e-6cdb-4735-b3fb-a8314f169786 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.970923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac55e-6cdb-4735-b3fb-a8314f169786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.971689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb38f4-1cb4-41cf-a91e-9a38e602d0ef None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.972053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb38f4-1cb4-41cf-a91e-9a38e602d0ef None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.972341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb38f4-1cb4-41cf-a91e-9a38e602d0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.000937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-99cac55e-6cdb-4735-b3fb-a8314f169786 None None] MFA Rules not processed for user `9c5ba513cf6b43c7b4908d6ef75ee493`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:50.002433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-99cac55e-6cdb-4735-b3fb-a8314f169786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.019314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-dfeb38f4-1cb4-41cf-a91e-9a38e602d0ef None None] MFA Rules not processed for user `f79a95f2221146a280a9d338c30be98a`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:50.020539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-dfeb38f4-1cb4-41cf-a91e-9a38e602d0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.099599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 804/1610] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:50.101659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cac55e-6cdb-4735-b3fb-a8314f169786 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.129125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cac55e-6cdb-4735-b3fb-a8314f169786 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.143395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d368d-b2de-4aa4-97ea-6e0155aa0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:50.146087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d368d-b2de-4aa4-97ea-6e0155aa0057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.146401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d368d-b2de-4aa4-97ea-6e0155aa0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.146695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d368d-b2de-4aa4-97ea-6e0155aa0057 admin admin] PATH_INFO: `/v3/projects/f43f79cc58f240188554263e21188737/users/3b5540dd03774ac3884c20084c2b4afa/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.169532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 807/1611] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:50.170414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb38f4-1cb4-41cf-a91e-9a38e602d0ef None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.173940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d368d-b2de-4aa4-97ea-6e0155aa0057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43f79cc58f240188554263e21188737, user_id=3b5540dd03774ac3884c20084c2b4afa, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:50.187387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb38f4-1cb4-41cf-a91e-9a38e602d0ef None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.191571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16309a91-c101-43ca-aa69-9dc5ddf23efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:50.191870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d368d-b2de-4aa4-97ea-6e0155aa0057 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:50.193464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16309a91-c101-43ca-aa69-9dc5ddf23efc service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.193464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16309a91-c101-43ca-aa69-9dc5ddf23efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.193464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16309a91-c101-43ca-aa69-9dc5ddf23efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.205865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16309a91-c101-43ca-aa69-9dc5ddf23efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:50.207709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16309a91-c101-43ca-aa69-9dc5ddf23efc service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:50.225160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 805/1612] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:50 2022] PUT /identity/v3/projects/f43f79cc58f240188554263e21188737/users/3b5540dd03774ac3884c20084c2b4afa/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:50.226935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce618-40bc-46c0-91be-61a117ffe0e5 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.228282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce618-40bc-46c0-91be-61a117ffe0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.228572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce618-40bc-46c0-91be-61a117ffe0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.239508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 808/1613] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:50.243484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595ed2-e06f-4f3a-831a-947a2e5ca73e None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.243782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595ed2-e06f-4f3a-831a-947a2e5ca73e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.244098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595ed2-e06f-4f3a-831a-947a2e5ca73e None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.246603 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 809/1614] 10.222.0.62 () {54 vars in 961 bytes} [Fri May 27 03:23:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:50.249793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf08cf9-5c84-43b7-a67c-60da4078911f None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.252437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf08cf9-5c84-43b7-a67c-60da4078911f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.252437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf08cf9-5c84-43b7-a67c-60da4078911f None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.254645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 810/1615] 10.222.0.62 () {54 vars in 961 bytes} [Fri May 27 03:23:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:50.256150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ca6f-4ab9-4bb3-87bc-87710b56d2d8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.256439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ca6f-4ab9-4bb3-87bc-87710b56d2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.256771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ca6f-4ab9-4bb3-87bc-87710b56d2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.289262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7020ca6f-4ab9-4bb3-87bc-87710b56d2d8 None None] MFA Rules not processed for user `3b5540dd03774ac3884c20084c2b4afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:50.291125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7020ca6f-4ab9-4bb3-87bc-87710b56d2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.295063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ca3ce618-40bc-46c0-91be-61a117ffe0e5 None None] MFA Rules not processed for user `0f04eeaae0a54cdea28b5a9760c1a864`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:50.296605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ca3ce618-40bc-46c0-91be-61a117ffe0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.411430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 811/1616] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:50.413464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8aed0-e643-4181-8dc1-ee8ee648153e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.413938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8aed0-e643-4181-8dc1-ee8ee648153e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.414551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8aed0-e643-4181-8dc1-ee8ee648153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.416037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:50.444684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b0f8aed0-e643-4181-8dc1-ee8ee648153e None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:50.446268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b0f8aed0-e643-4181-8dc1-ee8ee648153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.474696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 806/1617] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:50.477595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886191bc-9bb4-49be-844a-12b415ccf66f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.477595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886191bc-9bb4-49be-844a-12b415ccf66f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.477946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886191bc-9bb4-49be-844a-12b415ccf66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.514568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-886191bc-9bb4-49be-844a-12b415ccf66f None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:50.516520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-886191bc-9bb4-49be-844a-12b415ccf66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.578997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:50.611795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 812/1618] 10.222.0.62 () {58 vars in 1058 bytes} [Fri May 27 03:23:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:50.613035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8aed0-e643-4181-8dc1-ee8ee648153e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.641439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 807/1619] 10.222.0.62 () {58 vars in 1058 bytes} [Fri May 27 03:23:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:50.642380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886191bc-9bb4-49be-844a-12b415ccf66f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.704686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886191bc-9bb4-49be-844a-12b415ccf66f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.709265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:50.709927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.710211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.710487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.720657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:50.722403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:50.732792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 808/1620] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:50.733702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.744029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8aed0-e643-4181-8dc1-ee8ee648153e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.747887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde488-ce86-47cd-a92a-a7fa8ff244a5 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.753734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:50.754386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.754663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.754937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.755747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:50.757677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.758692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.759016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.761492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:50.767459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:50.824797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:50.826851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:50.841268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 813/1621] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:50 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:50.841731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 809/1622] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:50.841835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.842760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.868024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18755cd2-cf7a-45a9-ab82-9d3c8265b14e service nova] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.920519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:51.024266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.025163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.025546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.026250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.027133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.031605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.054085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f04f35-a89d-4b00-b0ca-7b1d836aedc9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.067533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 810/1623] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:50 2022] POST /identity/v3/users => generated 299 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:51.071252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.072422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.073374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.075059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.076137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.096632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1ac63-5b26-47b4-883c-870506bcd327 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.109076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.109934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.110765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.115391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.116370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.118527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.146158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 811/1624] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:51.150099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.170700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443546-7931-4f0f-8cb0-210dd4e6f9ea admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.180120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a4fa86-6069-4c73-9605-f8a4aa42bd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.181341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4fa86-6069-4c73-9605-f8a4aa42bd98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.181728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4fa86-6069-4c73-9605-f8a4aa42bd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.182133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4fa86-6069-4c73-9605-f8a4aa42bd98 admin admin] PATH_INFO: `/v3/projects/96f4997d9d8e4ec08a75323939ee07f6/users/e93d45dcaea14a0d85a1341abd9a002e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.235628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a4fa86-6069-4c73-9605-f8a4aa42bd98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f4997d9d8e4ec08a75323939ee07f6, user_id=e93d45dcaea14a0d85a1341abd9a002e, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.241739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.243026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a4fa86-6069-4c73-9605-f8a4aa42bd98 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.243566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.274930 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 814/1625] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:51.277300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 812/1626] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:51 2022] PUT /identity/v3/projects/96f4997d9d8e4ec08a75323939ee07f6/users/e93d45dcaea14a0d85a1341abd9a002e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:51.281658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.301798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8291-8836-4647-88fe-1ceb96d3f27d service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.318954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:51.439295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa44923-de55-4aa0-a207-018acf3e9fe6 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.439657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa44923-de55-4aa0-a207-018acf3e9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.440274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa44923-de55-4aa0-a207-018acf3e9fe6 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.441772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 813/1627] 10.222.0.62 () {54 vars in 912 bytes} [Fri May 27 03:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:51.454365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961444-2569-438c-91ca-fcb9341ac840 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.454651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961444-2569-438c-91ca-fcb9341ac840 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.454930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961444-2569-438c-91ca-fcb9341ac840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.462292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.462973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.463258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.463534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.464300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.465966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.505088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9a961444-2569-438c-91ca-fcb9341ac840 None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:51.506310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9a961444-2569-438c-91ca-fcb9341ac840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:51.516681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 815/1628] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:51.519919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.555805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce2b15-03c4-4ae2-a5c9-c7907932f491 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.569293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecba69-4c3a-4ccb-a8d4-a46958af5be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.570519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecba69-4c3a-4ccb-a8d4-a46958af5be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.571064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecba69-4c3a-4ccb-a8d4-a46958af5be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.571536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecba69-4c3a-4ccb-a8d4-a46958af5be0 admin admin] PATH_INFO: `/v3/projects/96f4997d9d8e4ec08a75323939ee07f6/users/e93d45dcaea14a0d85a1341abd9a002e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.575622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecba69-4c3a-4ccb-a8d4-a46958af5be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f4997d9d8e4ec08a75323939ee07f6, user_id=e93d45dcaea14a0d85a1341abd9a002e, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.580793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecba69-4c3a-4ccb-a8d4-a46958af5be0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.652270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 816/1629] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:23:51 2022] PUT /identity/v3/projects/96f4997d9d8e4ec08a75323939ee07f6/users/e93d45dcaea14a0d85a1341abd9a002e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:51.656892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efefbb0-76aa-4a80-b4c6-7e212c6da5b8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.657175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efefbb0-76aa-4a80-b4c6-7e212c6da5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.657451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efefbb0-76aa-4a80-b4c6-7e212c6da5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.702749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 814/1630] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:51.717295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a961444-2569-438c-91ca-fcb9341ac840 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.720726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2efefbb0-76aa-4a80-b4c6-7e212c6da5b8 None None] MFA Rules not processed for user `e93d45dcaea14a0d85a1341abd9a002e`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:51.722014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2efefbb0-76aa-4a80-b4c6-7e212c6da5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:51.838913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a961444-2569-438c-91ca-fcb9341ac840 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.866902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:51.968015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 817/1631] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:51.972646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453418a1-08fd-44a8-b694-3197d081082d None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.972646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453418a1-08fd-44a8-b694-3197d081082d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.972646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453418a1-08fd-44a8-b694-3197d081082d None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.975360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 818/1632] 10.222.0.62 () {54 vars in 912 bytes} [Fri May 27 03:23:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:51.983800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41608d3b-91e3-4e31-b349-d62b7e80c0cb None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.984246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41608d3b-91e3-4e31-b349-d62b7e80c0cb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.985464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41608d3b-91e3-4e31-b349-d62b7e80c0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.006059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.006739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.007044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.007291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.082010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-41608d3b-91e3-4e31-b349-d62b7e80c0cb None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:52.086506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-41608d3b-91e3-4e31-b349-d62b7e80c0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:52.210315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.213995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.239330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 819/1633] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:52.240338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41608d3b-91e3-4e31-b349-d62b7e80c0cb None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.248467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 815/1634] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:23:52.266174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41608d3b-91e3-4e31-b349-d62b7e80c0cb None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.280728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.282284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.283467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.283896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.300582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.302677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.310090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:52.324283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 820/1635] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:52.356672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.359193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.375460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3295f54-e6e6-47fa-a175-695ea5f64982 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.384625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9d402-4aa9-4950-9f90-1fa7c5eb7f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.385601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d402-4aa9-4950-9f90-1fa7c5eb7f5d service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.386566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d402-4aa9-4950-9f90-1fa7c5eb7f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.387054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d402-4aa9-4950-9f90-1fa7c5eb7f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.401852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9d402-4aa9-4950-9f90-1fa7c5eb7f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.404183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9d402-4aa9-4950-9f90-1fa7c5eb7f5d service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.434551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 821/1636] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:52.462170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828ad12-c17d-43bb-9ae5-2093ea82743c service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.468670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43398077-6859-498a-9014-67745cc76327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.469775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43398077-6859-498a-9014-67745cc76327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.470747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43398077-6859-498a-9014-67745cc76327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.471353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43398077-6859-498a-9014-67745cc76327 admin admin] PATH_INFO: `/v3/users/3b5540dd03774ac3884c20084c2b4afa` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.472907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43398077-6859-498a-9014-67745cc76327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5540dd03774ac3884c20084c2b4afa)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.475744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43398077-6859-498a-9014-67745cc76327 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.568263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5540dd03774ac3884c20084c2b4afa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.568792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5540dd03774ac3884c20084c2b4afa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.571770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5540dd03774ac3884c20084c2b4afa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.614673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5540dd03774ac3884c20084c2b4afa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.615778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:52.631059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5540dd03774ac3884c20084c2b4afa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.664206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 816/1637] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:23:52 2022] DELETE /identity/v3/users/3b5540dd03774ac3884c20084c2b4afa => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:52.670504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba65f1-3228-4b66-a690-89668aed8f65 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.672830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba65f1-3228-4b66-a690-89668aed8f65 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.673503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba65f1-3228-4b66-a690-89668aed8f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.738430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2bba65f1-3228-4b66-a690-89668aed8f65 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:52.740048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2bba65f1-3228-4b66-a690-89668aed8f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:52.904707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 822/1638] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:52.940006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.943471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:53.126194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43398077-6859-498a-9014-67745cc76327 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:53.149870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94178aa1-bca6-43a6-9f7f-1ff7e2eddc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:53.150290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94178aa1-bca6-43a6-9f7f-1ff7e2eddc7f service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:53.150575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94178aa1-bca6-43a6-9f7f-1ff7e2eddc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:53.150852 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94178aa1-bca6-43a6-9f7f-1ff7e2eddc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:53.182558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94178aa1-bca6-43a6-9f7f-1ff7e2eddc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:53.185990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94178aa1-bca6-43a6-9f7f-1ff7e2eddc7f service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:53.217785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 817/1639] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:55.129120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:55.130125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:55.130549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:55.197829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] MFA Rules not processed for user `e4d835d78d4f4b019ced356585af16c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:55.202621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:55.355853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 823/1640] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:55.363680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:55.364277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:55.364277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:55.408795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] MFA Rules not processed for user `e4d835d78d4f4b019ced356585af16c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:55.410053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:55.442014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 818/1641] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:55.455153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:55.458162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:55.665812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f0b68-c41e-47c4-baf9-c44e85d93307 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:55.679091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:55.682899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:55.683698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:55.685310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:55.710947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:55.716645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:55.734469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:55.763903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 824/1642] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:56.208364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:56.209701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:56.241069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecacb43-ec54-4cac-a34d-82697159f4d0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:56.256129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:56.257081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:56.257362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:56.257762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] PATH_INFO: `/v3/projects/f43f79cc58f240188554263e21188737` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:56.259123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43f79cc58f240188554263e21188737)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:56.260517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: WARNING py.warnings [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:56.260517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: warnings.warn(msg) May 27 03:23:56.260517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: May 27 03:23:56.261932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:56.411454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43f79cc58f240188554263e21188737', 'request_id': 'req-8c4aab53-7c96-4889-9f41-258edbccebf1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.428052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43f79cc58f240188554263e21188737', 'request_id': 'req-8c4aab53-7c96-4889-9f41-258edbccebf1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.437265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43f79cc58f240188554263e21188737', 'request_id': 'req-8c4aab53-7c96-4889-9f41-258edbccebf1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.462221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] The token cache is being invalidate because project f43f79cc58f240188554263e21188737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:56.462799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.463888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c4aab53-7c96-4889-9f41-258edbccebf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.466826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 819/1643] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:23:56 2022] DELETE /identity/v3/projects/f43f79cc58f240188554263e21188737 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:23:56.474374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:56.504917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d354b8-7212-4ed6-a9f0-f7bb5dbbe972 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:56.671707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:56.672651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:56.672937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:56.673207 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] PATH_INFO: `/v3/users/e93d45dcaea14a0d85a1341abd9a002e` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:56.679242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93d45dcaea14a0d85a1341abd9a002e)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:56.680411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: WARNING py.warnings [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:56.680411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: warnings.warn(msg) May 27 03:23:56.680411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: May 27 03:23:56.681640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:56.855180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93d45dcaea14a0d85a1341abd9a002e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.860127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93d45dcaea14a0d85a1341abd9a002e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.893923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93d45dcaea14a0d85a1341abd9a002e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.895476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93d45dcaea14a0d85a1341abd9a002e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.921766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56861343-7cf4-4f86-89ec-8dd5a242794c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93d45dcaea14a0d85a1341abd9a002e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:56.945164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 825/1644] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:23:56 2022] DELETE /identity/v3/users/e93d45dcaea14a0d85a1341abd9a002e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:56.950917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbd427-3413-4a27-a2a4-e1445368546a None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:56.951383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbd427-3413-4a27-a2a4-e1445368546a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:56.951867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbd427-3413-4a27-a2a4-e1445368546a None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:56.953482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 820/1645] 10.222.0.62 () {54 vars in 912 bytes} [Fri May 27 03:23:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:56.960248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:56.960248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:56.960491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:57.007095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] MFA Rules not processed for user `a364b6179abb4ad0883898090acef827`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:57.008381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:57.355562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 826/1646] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:23:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:57.408403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbd427-3413-4a27-a2a4-e1445368546a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:57.410505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-42bbd427-3413-4a27-a2a4-e1445368546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:57.438602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbd427-3413-4a27-a2a4-e1445368546a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:57.462823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:57.463560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:57.470015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:57.470530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:57.486843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:57.489068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:57.508200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 821/1647] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:58.096204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:58.098794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:58.295705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:58.462820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f38da0-24e7-4501-9ccd-a0041ccbae09 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:58.486898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:58.488622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.489126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.489613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] PATH_INFO: `/v3/projects/96f4997d9d8e4ec08a75323939ee07f6` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.491653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f4997d9d8e4ec08a75323939ee07f6)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:58.515104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:58.515621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:58.516731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:58.578335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e9a77-f4c9-4596-97eb-bbaa971cbf81 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:58.599190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:58.600966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.601940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.602398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] PATH_INFO: `/v3/users/9c5ba513cf6b43c7b4908d6ef75ee493` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.609802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c5ba513cf6b43c7b4908d6ef75ee493)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:58.638170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:58.696656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f4997d9d8e4ec08a75323939ee07f6', 'request_id': 'req-802494f3-b640-40ce-8d4f-6c57e43ca895'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.713479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f4997d9d8e4ec08a75323939ee07f6', 'request_id': 'req-802494f3-b640-40ce-8d4f-6c57e43ca895'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.745548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f4997d9d8e4ec08a75323939ee07f6', 'request_id': 'req-802494f3-b640-40ce-8d4f-6c57e43ca895'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.772561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] The token cache is being invalidate because project 96f4997d9d8e4ec08a75323939ee07f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:58.773126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.778191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-802494f3-b640-40ce-8d4f-6c57e43ca895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.780656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 827/1648] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:23:58 2022] DELETE /identity/v3/projects/96f4997d9d8e4ec08a75323939ee07f6 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:58.781950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da253e-fb8d-4211-aced-6d7a66284040 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.782267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da253e-fb8d-4211-aced-6d7a66284040 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.782551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da253e-fb8d-4211-aced-6d7a66284040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.812168 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:58.827762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c5ba513cf6b43c7b4908d6ef75ee493'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.829053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c5ba513cf6b43c7b4908d6ef75ee493'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.829706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c5ba513cf6b43c7b4908d6ef75ee493'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.838544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-44da253e-fb8d-4211-aced-6d7a66284040 None None] MFA Rules not processed for user `720bd38555404aafa93db5ae49b13acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:58.840053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-44da253e-fb8d-4211-aced-6d7a66284040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:58.867926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c5ba513cf6b43c7b4908d6ef75ee493'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.942339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d438a39a-eb9a-42fc-9273-1cc5d6533060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c5ba513cf6b43c7b4908d6ef75ee493'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.961624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 828/1649] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:58.963502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.965282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.965782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.977640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 822/1650] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:23:58 2022] DELETE /identity/v3/users/9c5ba513cf6b43c7b4908d6ef75ee493 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:23:58.982892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083d6f1-52c9-4fe6-9ba5-1937c32d0614 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.983366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083d6f1-52c9-4fe6-9ba5-1937c32d0614 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.983794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083d6f1-52c9-4fe6-9ba5-1937c32d0614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.031176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:59.045510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.046717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2083d6f1-52c9-4fe6-9ba5-1937c32d0614 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:59.048021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2083d6f1-52c9-4fe6-9ba5-1937c32d0614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.186822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:59.222568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 829/1651] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:59.222568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.222780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 823/1652] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:59.222863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e64c49-dfdc-44b9-ac0e-6e02fa9a8552 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.223094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e64c49-dfdc-44b9-ac0e-6e02fa9a8552 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.223374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e64c49-dfdc-44b9-ac0e-6e02fa9a8552 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.224911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 824/1653] 10.222.0.62 () {54 vars in 960 bytes} [Fri May 27 03:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:59.226400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.228009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbcbbd-c7e6-49b3-bbca-3b54fbd6e81a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.228299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbcbbd-c7e6-49b3-bbca-3b54fbd6e81a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.228576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbcbbd-c7e6-49b3-bbca-3b54fbd6e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.268831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d5dbcbbd-c7e6-49b3-bbca-3b54fbd6e81a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:59.270049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d5dbcbbd-c7e6-49b3-bbca-3b54fbd6e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.297143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 825/1654] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:59.304735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.305030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.305313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.341980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:59.343269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ea197-e87a-45a1-a5a2-a2f25d090e1c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:59.344026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.351004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:59.352182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.352671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.353165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.441025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:59.448418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:59.472237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 826/1655] 10.222.0.62 () {58 vars in 1057 bytes} [Fri May 27 03:23:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:59.472707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 830/1656] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:59.473239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.474924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.476237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.505852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5735-9bd0-4962-92e2-911138d1e629 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:59.517896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:59.518477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.519124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.519124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.519769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:59.520106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:59.521725 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:59.549649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 831/1657] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:59.550391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.568571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6047c-0e07-42a4-b69a-6b894af76d73 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:59.574073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3ad7-4389-4bd6-a129-f1a91d7f7381 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:59.576066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:59.577126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.580783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.581299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.585771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:59.586447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.586735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.587015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.588923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.591412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:59.597478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:59.600022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:59.618550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:59.623938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:59.639425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 827/1658] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:59.641107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.655941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 832/1659] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:23:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:59.668576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91d2ea-7ee4-4a30-bef9-40da1f360d26 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:59.705261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:59.709984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.710284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.711908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.712815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:59.716601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:59.787424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 828/1660] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:23:59 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:59.791430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.807783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37279f14-ede2-4fc0-8418-a03a4fa44cbb service placement] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.006478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.007245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.007541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.007913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.008552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.010557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.046503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:00.083704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 833/1661] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:23:59 2022] POST /identity/v3/users => generated 294 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:00.088098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:00.109390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d901d8-1b0e-45cb-a5c8-8bd584e3650f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.124540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.125247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.125620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.125911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.126669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.128524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.503568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 829/1662] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:00.507654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:00.520627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c361a50-8ff4-4cb3-b043-50eb7b57f1d0 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.540937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf1fec-cbc4-4b77-ba3a-d158632a3849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.541843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1fec-cbc4-4b77-ba3a-d158632a3849 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.542257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1fec-cbc4-4b77-ba3a-d158632a3849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.542674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1fec-cbc4-4b77-ba3a-d158632a3849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.544141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf1fec-cbc4-4b77-ba3a-d158632a3849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.545962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf1fec-cbc4-4b77-ba3a-d158632a3849 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.571393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 834/1663] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:00.577139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:00.590043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729661c-6ac0-43c2-bda6-bee408f0c6b6 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.602841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.605008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.605480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.606767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] PATH_INFO: `/v3/projects/a22c34163b30400b9eb194d47bc76eb3/users/09ae1366533e4623a7dd008be27df0b3/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.639069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22c34163b30400b9eb194d47bc76eb3, user_id=09ae1366533e4623a7dd008be27df0b3, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.647551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.682321 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 830/1664] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:00 2022] PUT /identity/v3/projects/a22c34163b30400b9eb194d47bc76eb3/users/09ae1366533e4623a7dd008be27df0b3/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:00.693697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f44e4-60ed-4d7e-ac62-04c3bac07ab7 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.694084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f44e4-60ed-4d7e-ac62-04c3bac07ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.694296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f44e4-60ed-4d7e-ac62-04c3bac07ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.791058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-228f44e4-60ed-4d7e-ac62-04c3bac07ab7 None None] MFA Rules not processed for user `09ae1366533e4623a7dd008be27df0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:00.792850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-228f44e4-60ed-4d7e-ac62-04c3bac07ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:00.982325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 835/1665] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:00.983839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:01.178572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68f03e-9085-42e8-bcb4-76d2bb8bbb86 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:01.208322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:01.209012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.209303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.209585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.210305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:01.212276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:01.311848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 831/1666] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:00 2022] POST /identity/v3/projects => generated 385 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:01.315674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f44e4-60ed-4d7e-ac62-04c3bac07ab7 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:01.334171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f44e4-60ed-4d7e-ac62-04c3bac07ab7 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:01.397793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:01.401095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:01.492748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:01.493376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.493742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.494030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.499091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:01.504422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:01.552263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 836/1667] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:01 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:01.552263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:01.552950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692576f-6cd4-43c2-927c-7799d5881862 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:01.555345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:01.561398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:01.562524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.562847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.563134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] PATH_INFO: `/v3/users/f79a95f2221146a280a9d338c30be98a` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.564448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79a95f2221146a280a9d338c30be98a)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:01.566463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:01.592994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f370e-765e-463b-b165-9375a2b0ddc8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:01.597358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:01.611117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:01.612176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.612510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.612849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] PATH_INFO: `/v3/projects/ac9d7226b54c494893714280b2a9be32` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.615746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9d7226b54c494893714280b2a9be32)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:01.617948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:01.646656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79a95f2221146a280a9d338c30be98a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.651699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79a95f2221146a280a9d338c30be98a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.653581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79a95f2221146a280a9d338c30be98a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.686148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79a95f2221146a280a9d338c30be98a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.705272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79a95f2221146a280a9d338c30be98a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.743226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 832/1668] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:01 2022] DELETE /identity/v3/users/f79a95f2221146a280a9d338c30be98a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:01.744609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:01.751200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:01.756840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9d7226b54c494893714280b2a9be32', 'request_id': 'req-8c5c0607-bb15-4c5c-ac25-f417a2219505'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.764223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:INSERT=6,keystone:SELECT=90,keystone:DELETE=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:01.778487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9d7226b54c494893714280b2a9be32', 'request_id': 'req-8c5c0607-bb15-4c5c-ac25-f417a2219505'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.793348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9d7226b54c494893714280b2a9be32', 'request_id': 'req-8c5c0607-bb15-4c5c-ac25-f417a2219505'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.824022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] The token cache is being invalidate because project ac9d7226b54c494893714280b2a9be32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:01.824377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.826377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8c5c0607-bb15-4c5c-ac25-f417a2219505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:01.833781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 837/1669] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:01 2022] DELETE /identity/v3/projects/ac9d7226b54c494893714280b2a9be32 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:01.833781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb87b26-7979-47f5-abd5-5123570016a4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.833781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb87b26-7979-47f5-abd5-5123570016a4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.835326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb87b26-7979-47f5-abd5-5123570016a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.882701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2bb87b26-7979-47f5-abd5-5123570016a4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:01.887935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2bb87b26-7979-47f5-abd5-5123570016a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:01.901015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ca3b-ae80-4e94-a341-f1be85570615 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:01.907790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 838/1670] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:01.908060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:01.909012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.909425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.909607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2686d-c7f0-4541-9ebb-52be272b691a None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.909892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.909998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2686d-c7f0-4541-9ebb-52be272b691a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.910185 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2686d-c7f0-4541-9ebb-52be272b691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.910899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:01.915213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:01.929609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 833/1671] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:01.932868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:01.936124 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:01.954023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84d8e6-cd21-4d2b-a165-b595f409ddd6 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:01.961067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:01.963083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:01.963083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:01.963468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:01.964495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:01.967366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:01.981275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d6a2686d-c7f0-4541-9ebb-52be272b691a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:01.982520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d6a2686d-c7f0-4541-9ebb-52be272b691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:01.983296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 834/1672] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:01.988086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.002690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 839/1673] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:02.005425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe421f-f096-4844-9f94-f6f1d4bcd65a None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.005717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe421f-f096-4844-9f94-f6f1d4bcd65a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.006003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe421f-f096-4844-9f94-f6f1d4bcd65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.007593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f058e3-30ae-49d5-93b2-c6eb1fafae66 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.013912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.015568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.016337 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.016784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] PATH_INFO: `/v3/projects/a01743fdb2874868a5239b7142b80ef0/users/b6cd9a8636e3481a8c4516c661da45e4/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.038033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a01743fdb2874868a5239b7142b80ef0, user_id=b6cd9a8636e3481a8c4516c661da45e4, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.041398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.057622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fcbe421f-f096-4844-9f94-f6f1d4bcd65a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:02.059786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fcbe421f-f096-4844-9f94-f6f1d4bcd65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.064611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 835/1674] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:01 2022] PUT /identity/v3/projects/a01743fdb2874868a5239b7142b80ef0/users/b6cd9a8636e3481a8c4516c661da45e4/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:02.065891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.067716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.167642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e937e-839a-4791-9705-91f3c857e4b1 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.172262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.173190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.173489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.173771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.196194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 840/1675] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:02.197485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.198727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645554ad-01fc-458b-8939-c684efd26564 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.198837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.200816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645554ad-01fc-458b-8939-c684efd26564 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.202407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645554ad-01fc-458b-8939-c684efd26564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.231520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 836/1676] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:02.232574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.241225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:02.250709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb073f-7f52-41d1-83aa-c6643f41fd12 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.268025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.268722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.269014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.269291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.270197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:02.270513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.276380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.284117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-645554ad-01fc-458b-8939-c684efd26564 None None] MFA Rules not processed for user `b6cd9a8636e3481a8c4516c661da45e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:02.289727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-645554ad-01fc-458b-8939-c684efd26564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.325703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 837/1677] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:02.328450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.347702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f8db0-d5b2-4b3f-aa3f-3362715d0699 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.363388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.364226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.364587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.364942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.365733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.371897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.452168 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 838/1678] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:02.456130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.486350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 841/1679] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:02.490615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645554ad-01fc-458b-8939-c684efd26564 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.525851 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645554ad-01fc-458b-8939-c684efd26564 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.629223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42651f-bce4-4351-814d-73732220ea74 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.647157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.647906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.648234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.648513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.649874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.654607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.674876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:02.699251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 839/1680] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:02.700793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.701580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.701914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.702423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.703093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.707463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.709029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.730123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8a20-3f87-46b9-a37d-f2fea9e116aa admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.739083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.740239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.740722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.742413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.743378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.746508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.774865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 842/1681] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:02.779489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.790356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 840/1682] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:02.794283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.813084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ff800-d19b-4e87-8bb9-e0096ba1120e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.996811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07e331-a047-4bfe-a301-17a2bb92a722 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.009620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.010363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.011381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.011381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.012582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.013051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.015589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.015900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.016187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.017281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.018369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.023212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.048224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 841/1683] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.051781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.065794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 843/1684] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:03.069584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.079787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919eebe-34be-42f6-bd05-444e8962bbe3 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.093198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ce0f9-ade2-4f42-83bc-c6eabdcbc8e8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.093646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9778b1b-c6af-4339-a964-ded324280fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.094661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9778b1b-c6af-4339-a964-ded324280fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.095046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9778b1b-c6af-4339-a964-ded324280fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.095850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9778b1b-c6af-4339-a964-ded324280fdf admin admin] PATH_INFO: `/v3/projects/e8cc4fcc64d3439e8fb3ab94927a37b1/users/d1d27f19405a4d3980910f783cbb8d16/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.118403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.119194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.119711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.127949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.128942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.132868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9778b1b-c6af-4339-a964-ded324280fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8cc4fcc64d3439e8fb3ab94927a37b1, user_id=d1d27f19405a4d3980910f783cbb8d16, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.138039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.139015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9778b1b-c6af-4339-a964-ded324280fdf admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.163521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 844/1685] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.163746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.181019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 842/1686] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:03 2022] PUT /identity/v3/projects/e8cc4fcc64d3439e8fb3ab94927a37b1/users/d1d27f19405a4d3980910f783cbb8d16/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:03.183074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334a00-0fbd-46f2-9e16-d158145b3429 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.183469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334a00-0fbd-46f2-9e16-d158145b3429 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.184791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334a00-0fbd-46f2-9e16-d158145b3429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.192669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7acac5-815f-4e28-a602-69fe44f9d3a4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.247493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-69334a00-0fbd-46f2-9e16-d158145b3429 None None] MFA Rules not processed for user `d1d27f19405a4d3980910f783cbb8d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:03.248837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-69334a00-0fbd-46f2-9e16-d158145b3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:03.272616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:03.364683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46289a-fa35-4298-809f-f15289dab18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.365621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46289a-fa35-4298-809f-f15289dab18f admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.368266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46289a-fa35-4298-809f-f15289dab18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.368606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46289a-fa35-4298-809f-f15289dab18f admin admin] PATH_INFO: `/v3/projects/fc217b00af4b4528ad7dbfab9e2d55b3/users/afba1f346f09487ba0dec42bf8cfff77/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.395352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 843/1687] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:03.397484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69334a00-0fbd-46f2-9e16-d158145b3429 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.421318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46289a-fa35-4298-809f-f15289dab18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc217b00af4b4528ad7dbfab9e2d55b3, user_id=afba1f346f09487ba0dec42bf8cfff77, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.425233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69334a00-0fbd-46f2-9e16-d158145b3429 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.426668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46289a-fa35-4298-809f-f15289dab18f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.441310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.442307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.442658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.443004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.443848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.446118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.466617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 845/1688] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:03 2022] PUT /identity/v3/projects/fc217b00af4b4528ad7dbfab9e2d55b3/users/afba1f346f09487ba0dec42bf8cfff77/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:03.471032 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127660-61d1-4e13-963c-0bd5a28b0477 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.471327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127660-61d1-4e13-963c-0bd5a28b0477 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.473772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127660-61d1-4e13-963c-0bd5a28b0477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.517693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 844/1689] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:03.518192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-6f127660-61d1-4e13-963c-0bd5a28b0477 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:03.518515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.521536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6f127660-61d1-4e13-963c-0bd5a28b0477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:03.628011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:03.673423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 846/1690] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:03.675141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f127660-61d1-4e13-963c-0bd5a28b0477 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.709589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d216fbb-1cd1-4d61-b777-437876d1c404 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.721392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.722205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.723801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.725864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.729965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.735067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.765915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 845/1691] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.771489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.795203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87837d0-157d-4d09-bdc9-c0528a70210a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.803976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5171ee-559a-4b53-b89d-882da7689d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.806350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5171ee-559a-4b53-b89d-882da7689d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.807486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5171ee-559a-4b53-b89d-882da7689d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.807922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5171ee-559a-4b53-b89d-882da7689d70 admin admin] PATH_INFO: `/v3/projects/fc217b00af4b4528ad7dbfab9e2d55b3/users/afba1f346f09487ba0dec42bf8cfff77/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.814395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5171ee-559a-4b53-b89d-882da7689d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc217b00af4b4528ad7dbfab9e2d55b3, user_id=afba1f346f09487ba0dec42bf8cfff77, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.822984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5171ee-559a-4b53-b89d-882da7689d70 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.889807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 846/1692] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:03 2022] PUT /identity/v3/projects/fc217b00af4b4528ad7dbfab9e2d55b3/users/afba1f346f09487ba0dec42bf8cfff77/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:03.891811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.898884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.899174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.950367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f127660-61d1-4e13-963c-0bd5a28b0477 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.966159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] MFA Rules not processed for user `afba1f346f09487ba0dec42bf8cfff77`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:03.967391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.168002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:04.202819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 847/1693] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:04.211897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.213874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.266980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.267925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.268330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.268724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.284184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.292593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.363348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 847/1694] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/users => generated 292 bytes in 687 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:04.364051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.374448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.422616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f82048-3760-4308-afd0-1d156a1d744f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.433056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.433783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.434067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.434340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.547142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e347eeb-1780-4510-94ce-47bfc509466c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.555186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.556829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.557242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.557638 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.571143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.575220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.582093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.587743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.603839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 848/1695] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:24:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:24:04.605508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.626455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 848/1696] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:04.640388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb423349-8a11-4f2f-8210-9dd220f2178c service swift] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.650148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.650852 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.651139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.651415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.656780 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.658482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.711165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 849/1697] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:04.716127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.740145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59164039-7750-41fe-a46a-99e3d4079bf3 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.757150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.762658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.766371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.780490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.780952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.781604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] PATH_INFO: `/v3/projects/0920e0cdd10f4c329aaf770df6b2ee2b/users/f3a5d4ca8b264dbf91e4eb9f71a94a3c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.811481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47877bbe-20e5-425e-9d30-001f88a8f962 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.856579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.859104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.862106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.865377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] PATH_INFO: `/v3/projects/7ba6e3acda8d4e41af12c7a79bf3bc3b` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.869874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba6e3acda8d4e41af12c7a79bf3bc3b)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.880351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.881487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0920e0cdd10f4c329aaf770df6b2ee2b, user_id=f3a5d4ca8b264dbf91e4eb9f71a94a3c, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.889956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.974314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 849/1698] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:04 2022] PUT /identity/v3/projects/0920e0cdd10f4c329aaf770df6b2ee2b/users/f3a5d4ca8b264dbf91e4eb9f71a94a3c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:04.975539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:05.001092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:05.153887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba6e3acda8d4e41af12c7a79bf3bc3b', 'request_id': 'req-884926e1-0333-4ff0-a388-6970be029852'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.173758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba6e3acda8d4e41af12c7a79bf3bc3b', 'request_id': 'req-884926e1-0333-4ff0-a388-6970be029852'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.196950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba6e3acda8d4e41af12c7a79bf3bc3b', 'request_id': 'req-884926e1-0333-4ff0-a388-6970be029852'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.223282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] The token cache is being invalidate because project 7ba6e3acda8d4e41af12c7a79bf3bc3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:05.224092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.225668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.235994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 850/1699] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:04 2022] DELETE /identity/v3/projects/7ba6e3acda8d4e41af12c7a79bf3bc3b => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:05.237147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:05.240848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:05.290380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:05.306033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ebd98-676e-4c65-95b1-2691ea999956 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:05.462273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884926e1-0333-4ff0-a388-6970be029852 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:05.477372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:05.478275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:05.478560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:05.478894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] PATH_INFO: `/v3/users/0f04eeaae0a54cdea28b5a9760c1a864` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:05.481264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f04eeaae0a54cdea28b5a9760c1a864)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:05.483240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:05.534499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7cedd-08e1-4ef1-baac-0c6b8f0d224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:05.541659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7cedd-08e1-4ef1-baac-0c6b8f0d224f admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:05.542146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7cedd-08e1-4ef1-baac-0c6b8f0d224f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:05.548226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7cedd-08e1-4ef1-baac-0c6b8f0d224f admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:05.549278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7cedd-08e1-4ef1-baac-0c6b8f0d224f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:05.551311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7cedd-08e1-4ef1-baac-0c6b8f0d224f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:05.580915 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 850/1700] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 606 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:05.582849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:05.594050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:05.594498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:05.598701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f04eeaae0a54cdea28b5a9760c1a864'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.599993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f04eeaae0a54cdea28b5a9760c1a864'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.601904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f04eeaae0a54cdea28b5a9760c1a864'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.647175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:05.656193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:05.680450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f04eeaae0a54cdea28b5a9760c1a864'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.682468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 851/1701] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:05.689790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:05.699852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:05.724795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71b76e25-79d6-4b1e-9980-963c4d811e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f04eeaae0a54cdea28b5a9760c1a864'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:05.742923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a23bd-0e72-4a6f-83a6-920cf3b683fe None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:05.763525 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 851/1702] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:05 2022] DELETE /identity/v3/users/0f04eeaae0a54cdea28b5a9760c1a864 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:05.773674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81611231-f4c5-45c2-8507-5c54e5d93c00 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:05.774055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81611231-f4c5-45c2-8507-5c54e5d93c00 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:05.774340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81611231-f4c5-45c2-8507-5c54e5d93c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:05.775107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7757d-a247-4df5-9ac9-2698c3d78b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:05.780664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7757d-a247-4df5-9ac9-2698c3d78b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:05.780993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7757d-a247-4df5-9ac9-2698c3d78b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:05.781278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7757d-a247-4df5-9ac9-2698c3d78b32 admin admin] PATH_INFO: `/v3/projects/0920e0cdd10f4c329aaf770df6b2ee2b/users/f3a5d4ca8b264dbf91e4eb9f71a94a3c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:05.785026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7757d-a247-4df5-9ac9-2698c3d78b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0920e0cdd10f4c329aaf770df6b2ee2b, user_id=f3a5d4ca8b264dbf91e4eb9f71a94a3c, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:05.787756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7757d-a247-4df5-9ac9-2698c3d78b32 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:05.828121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 852/1703] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:05 2022] PUT /identity/v3/projects/0920e0cdd10f4c329aaf770df6b2ee2b/users/f3a5d4ca8b264dbf91e4eb9f71a94a3c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:05.836953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df2ff7-f7fa-4b4e-b300-2581bf02bfdc None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:05.836953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df2ff7-f7fa-4b4e-b300-2581bf02bfdc None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:05.836953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df2ff7-f7fa-4b4e-b300-2581bf02bfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:05.850264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-81611231-f4c5-45c2-8507-5c54e5d93c00 None None] MFA Rules not processed for user `e8fddb1cf74d4135b41000f8a6293020`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:05.852976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-81611231-f4c5-45c2-8507-5c54e5d93c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:05.900264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-04df2ff7-f7fa-4b4e-b300-2581bf02bfdc None None] MFA Rules not processed for user `720bd38555404aafa93db5ae49b13acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:05.901663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-04df2ff7-f7fa-4b4e-b300-2581bf02bfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.085175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 852/1704] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:06.086716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5209e-04b7-437e-beba-e7ad4576a28d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:06.086929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5209e-04b7-437e-beba-e7ad4576a28d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:06.087220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5209e-04b7-437e-beba-e7ad4576a28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:06.134042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 853/1705] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:06.140773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6812-5f66-46f9-8915-7a31fa47f5d6 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:06.141068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6812-5f66-46f9-8915-7a31fa47f5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:06.141525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6812-5f66-46f9-8915-7a31fa47f5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:06.165902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-81d5209e-04b7-437e-beba-e7ad4576a28d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:06.167890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-81d5209e-04b7-437e-beba-e7ad4576a28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.208680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-33bf6812-5f66-46f9-8915-7a31fa47f5d6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:06.216379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:06.217746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-33bf6812-5f66-46f9-8915-7a31fa47f5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.356686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:06.453527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 853/1706] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:06.458457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f50dc-86b7-46c0-bf59-084bd22f3255 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:06.459442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f50dc-86b7-46c0-bf59-084bd22f3255 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:06.460469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f50dc-86b7-46c0-bf59-084bd22f3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:06.512849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 854/1707] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:06.518843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:06.520120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:06.520704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:06.553718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-eb6f50dc-86b7-46c0-bf59-084bd22f3255 None None] MFA Rules not processed for user `f3a5d4ca8b264dbf91e4eb9f71a94a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:06.554937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-eb6f50dc-86b7-46c0-bf59-084bd22f3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.559902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] MFA Rules not processed for user `e8fddb1cf74d4135b41000f8a6293020`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:06.561784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.582228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 855/1708] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:06.587646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:06.592576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.756375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d75a77-fda3-4d38-b25c-f43abf51b2da None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:06.779036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 854/1709] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:06.781001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f50dc-86b7-46c0-bf59-084bd22f3255 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:06.791183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:06.794708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:06.795809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:06.797448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:06.806208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.807546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f50dc-86b7-46c0-bf59-084bd22f3255 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:06.821916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:06.822912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:06.823454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:06.824166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:06.825295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:06.826034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:06.827773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:06.865815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 855/1710] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:06.867182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:06.872515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:06.997193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:06.998059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:07.031372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 856/1711] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:07.040805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:07.045106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:07.100396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e352255-07eb-4631-8503-15f9ac44f7b4 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:07.105359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:07.107030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:07.107442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:07.107875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:07.127611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:07.130748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:07.172799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 856/1712] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:07.174513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:07.178821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:07.237854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32196a3-ce6a-4a3b-9752-f6e6dd8fcc56 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:07.253924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:07.259455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:07.259740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:07.261432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:07.294162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:07.300192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:07.330656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055256d-b18b-45f5-a487-93ddb14d9569 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:07.349848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1664f9-8963-40fd-9175-695f5331ca6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:07.351581 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1664f9-8963-40fd-9175-695f5331ca6e service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:07.352614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1664f9-8963-40fd-9175-695f5331ca6e service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:07.353954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1664f9-8963-40fd-9175-695f5331ca6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:07.361530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 857/1713] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:07.362749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:07.384792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1664f9-8963-40fd-9175-695f5331ca6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:07.389965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1664f9-8963-40fd-9175-695f5331ca6e service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:07.424067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218c9f1-cbe7-448e-b659-2ce18885d225 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:07.446055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:07.447786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 857/1714] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:07.449310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:07.449658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:07.449916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:07.451359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:07.451769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:07.452080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:07.458197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:07.472156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:07.495346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:07.515080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] MFA Rules not processed for user `a364b6179abb4ad0883898090acef827`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:07.516897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:07.541825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 858/1715] 10.222.0.60 () {58 vars in 1015 bytes} [Fri May 27 03:24:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:07.560177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:07.561605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:07.580103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:07.581357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac90a84-0e65-4324-9938-f5778943f43c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:07.596547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:07.597220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:07.597503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:07.597777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:07.614656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:07.618460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:07.622918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 858/1716] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:07 2022] POST /identity/v3/projects => generated 363 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:07.623948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:07.642437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 859/1717] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:07.689095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:07.701542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cbe33-60b4-4bbe-a573-ed02a62f2c5b service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:07.835973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:07.836765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:07.836976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:07.843048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:07.847278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:07.978719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e7d83-3fc9-49fa-b3c1-89e949b9fafc admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:07.998544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:07.999301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:08.004282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:08.004732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:08.005472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:08.007420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:08.012106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:08.015301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:08.050983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 860/1718] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:08.121790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:08.131447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 859/1719] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:07 2022] POST /identity/v3/users => generated 282 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:08.167333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1905d1-d79e-4bcf-9d33-3b58931dc3de service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:08.208904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:08.209631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:08.209924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:08.210208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:08.210973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:08.217764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:08.273596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 861/1720] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:08.278482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:08.303027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2fb2-a993-4721-b8fe-f56c8dd5b07d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:08.316407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8e9bc-6000-4011-9960-a1a99a1b579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:08.317100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8e9bc-6000-4011-9960-a1a99a1b579b admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:08.317384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8e9bc-6000-4011-9960-a1a99a1b579b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:08.317715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8e9bc-6000-4011-9960-a1a99a1b579b admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:08.318708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8e9bc-6000-4011-9960-a1a99a1b579b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:08.320610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8e9bc-6000-4011-9960-a1a99a1b579b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:08.365586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 860/1721] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:08.370532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:08.426519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591136bf-4f32-4aed-b8d5-ae8ae791f57b admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:08.452116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:08.455033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:08.458110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:08.459184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] PATH_INFO: `/v3/projects/3b0ca4ed871d46be9bb293eae9f8b5f9/users/8cda70c9df71418fb92860f03f4e6fda/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:08.558578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0ca4ed871d46be9bb293eae9f8b5f9, user_id=8cda70c9df71418fb92860f03f4e6fda, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:08.575775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:08.632077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 862/1722] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:08 2022] PUT /identity/v3/projects/3b0ca4ed871d46be9bb293eae9f8b5f9/users/8cda70c9df71418fb92860f03f4e6fda/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:08.645600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e023675-3c39-440f-a093-48fbe52f1c63 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:08.646198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e023675-3c39-440f-a093-48fbe52f1c63 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:08.646769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e023675-3c39-440f-a093-48fbe52f1c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:08.705594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8e023675-3c39-440f-a093-48fbe52f1c63 None None] MFA Rules not processed for user `8cda70c9df71418fb92860f03f4e6fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:08.707301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8e023675-3c39-440f-a093-48fbe52f1c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:08.856949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:08.861977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 861/1723] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:08 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:09.004780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0cb0e-d3c1-43d6-bc9d-923d40285b49 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:09.017412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f36c3e-c9e3-4bcb-8f8b-b38a730d2905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:09.018097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36c3e-c9e3-4bcb-8f8b-b38a730d2905 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:09.018554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36c3e-c9e3-4bcb-8f8b-b38a730d2905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:09.018994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36c3e-c9e3-4bcb-8f8b-b38a730d2905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:09.019880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f36c3e-c9e3-4bcb-8f8b-b38a730d2905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:09.022149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f36c3e-c9e3-4bcb-8f8b-b38a730d2905 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:09.041710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:09.090231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e023675-3c39-440f-a093-48fbe52f1c63 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:09.093890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 863/1724] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:08 2022] POST /identity/v3/projects => generated 363 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:09.118860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e023675-3c39-440f-a093-48fbe52f1c63 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:09.246393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aac251-53dd-49f4-bb2f-9098267eb1ef None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:09.246688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aac251-53dd-49f4-bb2f-9098267eb1ef None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:09.246973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aac251-53dd-49f4-bb2f-9098267eb1ef None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:09.249258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 864/1725] 10.222.0.62 () {54 vars in 912 bytes} [Fri May 27 03:24:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:24:09.267298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5bc47-1e70-4655-9fa9-aee80c5cc892 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:09.267764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5bc47-1e70-4655-9fa9-aee80c5cc892 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:09.268168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5bc47-1e70-4655-9fa9-aee80c5cc892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:09.286593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:09.287371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:09.287695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:09.288000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:09.288721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:09.299532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:09.311891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-eab5bc47-1e70-4655-9fa9-aee80c5cc892 None None] MFA Rules not processed for user `a364b6179abb4ad0883898090acef827`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:09.320579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:09.321459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-eab5bc47-1e70-4655-9fa9-aee80c5cc892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:09.351445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 862/1726] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:09 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:09.352430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:09.377866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7af8-df21-419c-a400-1b34b8e1334e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:09.404844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:09.405536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:09.405818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:09.406095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:09.406844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:09.408502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:09.459439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 863/1727] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:09.474120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:09.532510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe507da6-0d3b-463a-a9d1-52882dad776b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:09.562738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:09.564228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:09.572099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:09.572556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] PATH_INFO: `/v3/projects/4e4e51e779dd4d628573d3a22a5b693f/users/2d834a989ba84db9880463500fec06ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:09.574009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 865/1728] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:24:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:09.638547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4e51e779dd4d628573d3a22a5b693f, user_id=2d834a989ba84db9880463500fec06ec, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:09.641926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:09.682682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 864/1729] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:09 2022] PUT /identity/v3/projects/4e4e51e779dd4d628573d3a22a5b693f/users/2d834a989ba84db9880463500fec06ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:09.686441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5bc47-1e70-4655-9fa9-aee80c5cc892 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:09.704624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5bc47-1e70-4655-9fa9-aee80c5cc892 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:09.807252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:09.929552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:09.938329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:09.940701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:09.941167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:09.942082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:09.955702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.010671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 866/1730] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:10.020334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:10.033148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386d8ec-c2b9-41e0-9f4f-e33a6f83f3ca admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.044275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4a50a-7110-491e-bdc9-4df2dab26748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.045088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4a50a-7110-491e-bdc9-4df2dab26748 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.045497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4a50a-7110-491e-bdc9-4df2dab26748 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.045883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4a50a-7110-491e-bdc9-4df2dab26748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.061488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4a50a-7110-491e-bdc9-4df2dab26748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.065135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4a50a-7110-491e-bdc9-4df2dab26748 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.082551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bf781-f0a7-497b-9dfb-d5ddc86869c5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.086837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 865/1731] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:10.113430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.115360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.115655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.118569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] PATH_INFO: `/v3/projects/4e4e51e779dd4d628573d3a22a5b693f/users/2d834a989ba84db9880463500fec06ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.144320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4e51e779dd4d628573d3a22a5b693f, user_id=2d834a989ba84db9880463500fec06ec, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.152310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.205987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 867/1732] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:10 2022] PUT /identity/v3/projects/4e4e51e779dd4d628573d3a22a5b693f/users/2d834a989ba84db9880463500fec06ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:10.217517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38808f7c-4917-411c-b68e-9b12661eb330 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.218613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38808f7c-4917-411c-b68e-9b12661eb330 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.219097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38808f7c-4917-411c-b68e-9b12661eb330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.335477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-38808f7c-4917-411c-b68e-9b12661eb330 None None] MFA Rules not processed for user `2d834a989ba84db9880463500fec06ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:10.339474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-38808f7c-4917-411c-b68e-9b12661eb330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:10.753457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 866/1733] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:10 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 537 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:10.924620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:11.251962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8a5ef-60e0-4977-9987-f6da99b5c12a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:11.270015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8cb99-d0eb-4a9c-8433-cd2932bd9f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:11.277663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8cb99-d0eb-4a9c-8433-cd2932bd9f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:11.279920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8cb99-d0eb-4a9c-8433-cd2932bd9f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:11.281257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8cb99-d0eb-4a9c-8433-cd2932bd9f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:11.349733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8cb99-d0eb-4a9c-8433-cd2932bd9f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:11.374263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8cb99-d0eb-4a9c-8433-cd2932bd9f4e service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:11.426439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 868/1734] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:10 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:11.656929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38808f7c-4917-411c-b68e-9b12661eb330 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:11.669764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38808f7c-4917-411c-b68e-9b12661eb330 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:11.683178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa539c-b52e-4969-9745-a0b5ffee1082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:11.683878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa539c-b52e-4969-9745-a0b5ffee1082 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:11.684165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa539c-b52e-4969-9745-a0b5ffee1082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:11.684446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa539c-b52e-4969-9745-a0b5ffee1082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:11.803803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c793-6d5d-44bb-9a9a-af28bb915015 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:11.804115 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c793-6d5d-44bb-9a9a-af28bb915015 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:11.804399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c793-6d5d-44bb-9a9a-af28bb915015 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:11.806154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 869/1735] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:24:11.929473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76929f-203b-4911-bb2d-09e52d278a6c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:11.929982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76929f-203b-4911-bb2d-09e52d278a6c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:11.938270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76929f-203b-4911-bb2d-09e52d278a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:11.967899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:12.026371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0b76929f-203b-4911-bb2d-09e52d278a6c None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:12.040569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0b76929f-203b-4911-bb2d-09e52d278a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:12.047217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa539c-b52e-4969-9745-a0b5ffee1082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:12.060513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa539c-b52e-4969-9745-a0b5ffee1082 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:12.114689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 867/1736] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:11 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:12.133740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:12.170040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:12.170336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:12.170672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:12.260136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] MFA Rules not processed for user `e8fddb1cf74d4135b41000f8a6293020`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:12.266049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:12.277133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 870/1737] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:24:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:12.349184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e822a9-a951-4175-8174-2acb6c99428c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:12.350018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e822a9-a951-4175-8174-2acb6c99428c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:12.350444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e822a9-a951-4175-8174-2acb6c99428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:12.403697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-99e822a9-a951-4175-8174-2acb6c99428c None None] MFA Rules not processed for user `b34132c9382a44cd97bc2c4e40c97eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:12.405330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-99e822a9-a951-4175-8174-2acb6c99428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:12.620604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 868/1738] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:12.623338 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:12.640721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:12.746920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 871/1739] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:12.748123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e822a9-a951-4175-8174-2acb6c99428c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:12.813009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e822a9-a951-4175-8174-2acb6c99428c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:12.832925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5bef0-c505-46d5-a500-af78271dae34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:12.834394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5bef0-c505-46d5-a500-af78271dae34 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:12.834826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5bef0-c505-46d5-a500-af78271dae34 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:12.835250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5bef0-c505-46d5-a500-af78271dae34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:12.859478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5bef0-c505-46d5-a500-af78271dae34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:12.872748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5bef0-c505-46d5-a500-af78271dae34 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:12.906714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 872/1740] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:12.910625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befc2f7-fac5-4778-970e-7facb9d62af6 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:12.926049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:12.927414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:12.927972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:12.928484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] PATH_INFO: `/v3/projects/9943a29a4d00458983620f79e7e07765` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:12.931001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9943a29a4d00458983620f79e7e07765)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:12.940608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:13.201469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9943a29a4d00458983620f79e7e07765', 'request_id': 'req-59af6e90-1104-4e64-b522-3201ff137111'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:13.217272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9943a29a4d00458983620f79e7e07765', 'request_id': 'req-59af6e90-1104-4e64-b522-3201ff137111'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:13.226812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9943a29a4d00458983620f79e7e07765', 'request_id': 'req-59af6e90-1104-4e64-b522-3201ff137111'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:13.254499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] The token cache is being invalidate because project 9943a29a4d00458983620f79e7e07765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:13.255167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:13.257242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-59af6e90-1104-4e64-b522-3201ff137111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:13.262685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 869/1741] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:12 2022] DELETE /identity/v3/projects/9943a29a4d00458983620f79e7e07765 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:13.268726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:13.269030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:13.269442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:13.339931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:13.344787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:13.588005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332950d2-5620-496a-9924-4be2afc7c4ea None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:13.588373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 873/1742] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:13.588793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332950d2-5620-496a-9924-4be2afc7c4ea None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:13.588994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332950d2-5620-496a-9924-4be2afc7c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:13.645559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-332950d2-5620-496a-9924-4be2afc7c4ea None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:13.647585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-332950d2-5620-496a-9924-4be2afc7c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:13.687157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:13.688890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:13.713479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 870/1743] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:13.719380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332950d2-5620-496a-9924-4be2afc7c4ea None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:13.746191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332950d2-5620-496a-9924-4be2afc7c4ea None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:13.781308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:13.791698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:13.792793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:13.793239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:13.793689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:13.794594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:13.795053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:13.796903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:13.837401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4a119-5543-4b70-8d57-ba1dda3be680 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:13.841127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 871/1744] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:13.842154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:13.848815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:13.858743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36daf9-38c4-41db-b789-7a441c5cc267 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:13.859081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:13.859778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:13.860078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:13.860359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:13.862285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:13.876512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:13.877469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:13.878771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:13.879163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:13.881111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:13.888674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:13.963334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 872/1745] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:13 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:13.965616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.051632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:14.053654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:14.080498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 874/1746] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:14.123023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e744889-5b16-489f-a154-addbf12f0079 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:14.130314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.131874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:14.137296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:14.138005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.140061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.140360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.142168 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:14.147063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:14.189256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 873/1747] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:13 2022] POST /identity/v3/users => generated 302 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:14.192963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.205749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba83a8-b31d-40bb-8405-3b8101fe4a1a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:14.213585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:14.214824 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.215157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.215448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.216722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:14.218352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:14.233087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 874/1748] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:14.237430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.253660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7db6e-73ee-42ff-9490-45779025f1a9 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:14.263135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:14.266312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.266312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.268548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.271810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:14.276455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:14.304744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 875/1749] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:14.306915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.341978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4babad9-c645-41b3-89ff-176b0bce5d21 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:14.353063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841e877-64d8-4bbb-b123-c502191aa168 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:14.357697 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d6bc-383f-42cd-86eb-263d9d51914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:14.358899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d6bc-383f-42cd-86eb-263d9d51914b admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.359261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d6bc-383f-42cd-86eb-263d9d51914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.360413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d6bc-383f-42cd-86eb-263d9d51914b admin admin] PATH_INFO: `/v3/projects/369f9c0c4b714bc982134dc9db9ba237/users/22edad0496ce4464882901f4733d76bd/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.430944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d6bc-383f-42cd-86eb-263d9d51914b admin admin] RBAC: Authorizing `identity:create_grant(project_id=369f9c0c4b714bc982134dc9db9ba237, user_id=22edad0496ce4464882901f4733d76bd, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:14.459856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d6bc-383f-42cd-86eb-263d9d51914b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:14.501662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:14.502617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.503005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.503384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.506226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:14.531318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 876/1750] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:14 2022] PUT /identity/v3/projects/369f9c0c4b714bc982134dc9db9ba237/users/22edad0496ce4464882901f4733d76bd/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:14.538016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e4d8-21f4-4a18-b6d2-66eb369a4ba9 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.538475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e4d8-21f4-4a18-b6d2-66eb369a4ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.538926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e4d8-21f4-4a18-b6d2-66eb369a4ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.547021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:14.588977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e2f5e4d8-21f4-4a18-b6d2-66eb369a4ba9 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:14.590467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e2f5e4d8-21f4-4a18-b6d2-66eb369a4ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:14.607568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:14.610070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:14.621853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 875/1751] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:14.801093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 877/1752] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:14.811874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.813704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:14.993467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db8ee9-5cd5-45cd-abd0-dc8d3a2acd82 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:15.082330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:15.083321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:15.083518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:15.084001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:15.095602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:15.097637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:15.124201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 876/1753] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:17.237240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170b713-2f4b-4010-b128-69297b870f99 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.237823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170b713-2f4b-4010-b128-69297b870f99 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.237823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170b713-2f4b-4010-b128-69297b870f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.312759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b170b713-2f4b-4010-b128-69297b870f99 None None] MFA Rules not processed for user `22edad0496ce4464882901f4733d76bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:17.315523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b170b713-2f4b-4010-b128-69297b870f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:17.362336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:17.481830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 878/1754] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:17.485194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:17.500056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e32669-e566-4f85-b76c-5e5d1d6149a9 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:17.510436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:17.511685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.513994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.514300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.515136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:17.521350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:17.583143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 877/1755] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:17.587050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170b713-2f4b-4010-b128-69297b870f99 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:17.602510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170b713-2f4b-4010-b128-69297b870f99 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:17.886082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:17.887804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.904459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.904459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.905010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:17.907169 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.001433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.001628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 879/1756] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/users => generated 309 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:18.011965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac4faa-422d-4f1c-b8e4-579b3d57d0ac admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.041973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.043475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.046249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.046749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.047679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.049466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.091742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 878/1757] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:18.096089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.112066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263b966-0f9f-4f9c-8698-fe68e64f1c00 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.125736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.126696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.126988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.127262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] PATH_INFO: `/v3/projects/7fb3a20ea17548909fa7dc28b01e3711/users/5e17d3569e394ba180663d67a05bfe60/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.168946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb3a20ea17548909fa7dc28b01e3711, user_id=5e17d3569e394ba180663d67a05bfe60, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.171723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.238960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 880/1758] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:18 2022] PUT /identity/v3/projects/7fb3a20ea17548909fa7dc28b01e3711/users/5e17d3569e394ba180663d67a05bfe60/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:18.242864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.273974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da8bbc-f9e8-4471-8393-4ac0f14e8a08 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.458014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:18.474691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062c2ab-711c-4a6c-b390-0370038279cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.475370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062c2ab-711c-4a6c-b390-0370038279cc admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.475655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062c2ab-711c-4a6c-b390-0370038279cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.475963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062c2ab-711c-4a6c-b390-0370038279cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.476785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062c2ab-711c-4a6c-b390-0370038279cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.478482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062c2ab-711c-4a6c-b390-0370038279cc admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.510373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 879/1759] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:18.513291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.531923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddefb3-f266-43a3-afda-0cd473b3f748 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.572448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.574700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.575047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.575317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] PATH_INFO: `/v3/projects/7fb3a20ea17548909fa7dc28b01e3711/users/5e17d3569e394ba180663d67a05bfe60/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.581831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb3a20ea17548909fa7dc28b01e3711, user_id=5e17d3569e394ba180663d67a05bfe60, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.588573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.631740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 881/1760] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:18 2022] PUT /identity/v3/projects/7fb3a20ea17548909fa7dc28b01e3711/users/5e17d3569e394ba180663d67a05bfe60/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:19.632688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e006866-ad2d-42fc-94e3-3791720b788d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.633588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e006866-ad2d-42fc-94e3-3791720b788d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.634005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e006866-ad2d-42fc-94e3-3791720b788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.676692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-1e006866-ad2d-42fc-94e3-3791720b788d None None] MFA Rules not processed for user `b34132c9382a44cd97bc2c4e40c97eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:19.683809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1e006866-ad2d-42fc-94e3-3791720b788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.937729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 880/1761] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:20.264815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:20.266658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:20.356153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b32e5c-ac3e-41be-9902-a3d3d555df42 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.362615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.363802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.364248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.364668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] PATH_INFO: `/v3/users/09ae1366533e4623a7dd008be27df0b3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.366024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ae1366533e4623a7dd008be27df0b3)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.368092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.380114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:20.442928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ae1366533e4623a7dd008be27df0b3'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:20.443989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ae1366533e4623a7dd008be27df0b3'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:20.444619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ae1366533e4623a7dd008be27df0b3'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:20.460038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ae1366533e4623a7dd008be27df0b3'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:20.469410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ae1366533e4623a7dd008be27df0b3'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:20.496886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 882/1762] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:20 2022] DELETE /identity/v3/users/09ae1366533e4623a7dd008be27df0b3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:20.500164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33739db7-0709-4f17-affd-9789605371e0 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.500460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33739db7-0709-4f17-affd-9789605371e0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.500745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33739db7-0709-4f17-affd-9789605371e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.527847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-33739db7-0709-4f17-affd-9789605371e0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:20.530871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-33739db7-0709-4f17-affd-9789605371e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:20.614564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 881/1763] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:20.629271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:20.630728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:20.704877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63fc21-4f21-4edb-9147-c4a313b148d9 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.709326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.710287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.710580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.710980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.723992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.725730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.740860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 883/1764] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:21.057630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc256-2a22-4b3a-ac06-04f5c7aa054e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.058119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc256-2a22-4b3a-ac06-04f5c7aa054e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.058216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc256-2a22-4b3a-ac06-04f5c7aa054e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.103029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-cdcdc256-2a22-4b3a-ac06-04f5c7aa054e None None] MFA Rules not processed for user `5e17d3569e394ba180663d67a05bfe60`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:21.105044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-cdcdc256-2a22-4b3a-ac06-04f5c7aa054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.221383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 882/1765] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:21.281307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:21.282882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.297207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d381fc1-c1d1-4c52-8a45-0543d6ba505d service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:21.303205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:21.304177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.304473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.304853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] PATH_INFO: `/v3/projects/a22c34163b30400b9eb194d47bc76eb3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.305986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22c34163b30400b9eb194d47bc76eb3)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:21.308222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:21.382366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22c34163b30400b9eb194d47bc76eb3', 'request_id': 'req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.396466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22c34163b30400b9eb194d47bc76eb3', 'request_id': 'req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.405095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22c34163b30400b9eb194d47bc76eb3', 'request_id': 'req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.418438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] The token cache is being invalidate because project a22c34163b30400b9eb194d47bc76eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:21.419156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.420739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.422722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 884/1766] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:21 2022] DELETE /identity/v3/projects/a22c34163b30400b9eb194d47bc76eb3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:21.426909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc256-2a22-4b3a-ac06-04f5c7aa054e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:21.445019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc256-2a22-4b3a-ac06-04f5c7aa054e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:21.460655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:21.525319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:21.526874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.527275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.527625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] PATH_INFO: `/v3/users/b6cd9a8636e3481a8c4516c661da45e4` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.528957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6cd9a8636e3481a8c4516c661da45e4)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:21.530898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:21.591176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6cd9a8636e3481a8c4516c661da45e4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.592097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6cd9a8636e3481a8c4516c661da45e4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.604788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6cd9a8636e3481a8c4516c661da45e4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.605489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6cd9a8636e3481a8c4516c661da45e4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.619332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6cd9a8636e3481a8c4516c661da45e4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.632128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 883/1767] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:21 2022] DELETE /identity/v3/users/b6cd9a8636e3481a8c4516c661da45e4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:21.915927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:21.917408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.930492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feb8b5-6442-4676-9f6a-5b61bedcdaca admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:21.939451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:21.992373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:21.993267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.993567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.993830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] PATH_INFO: `/v3/projects/a01743fdb2874868a5239b7142b80ef0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.995585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a01743fdb2874868a5239b7142b80ef0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:21.997635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:22.043064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a01743fdb2874868a5239b7142b80ef0', 'request_id': 'req-e03acd31-ee1c-434c-a28c-cdfbc89ea297'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.050581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a01743fdb2874868a5239b7142b80ef0', 'request_id': 'req-e03acd31-ee1c-434c-a28c-cdfbc89ea297'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.056199 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a01743fdb2874868a5239b7142b80ef0', 'request_id': 'req-e03acd31-ee1c-434c-a28c-cdfbc89ea297'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.070997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] The token cache is being invalidate because project a01743fdb2874868a5239b7142b80ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:22.071709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.071984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e03acd31-ee1c-434c-a28c-cdfbc89ea297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.077501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 885/1768] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:21 2022] DELETE /identity/v3/projects/a01743fdb2874868a5239b7142b80ef0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:22.078676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:22.080374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:22.095047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958c0b0-fbbd-4a52-84b2-5961d595c3af admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:22.202359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:22.204822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:22.205771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:22.207589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] PATH_INFO: `/v3/users/afba1f346f09487ba0dec42bf8cfff77` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:22.210095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=afba1f346f09487ba0dec42bf8cfff77)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:22.214104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:22.312362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:22.332427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afba1f346f09487ba0dec42bf8cfff77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.332427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afba1f346f09487ba0dec42bf8cfff77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.351809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afba1f346f09487ba0dec42bf8cfff77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.362938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873049-8001-4148-b2e9-814db691ea21 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:22.363618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873049-8001-4148-b2e9-814db691ea21 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:22.363618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873049-8001-4148-b2e9-814db691ea21 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:22.366087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 886/1769] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:24:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:24:22.380852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afba1f346f09487ba0dec42bf8cfff77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.381382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:22.381673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:22.381948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:22.398442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afba1f346f09487ba0dec42bf8cfff77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.402878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:22.404330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:22.417314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 884/1770] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:22 2022] DELETE /identity/v3/users/afba1f346f09487ba0dec42bf8cfff77 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:22.524021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 887/1771] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:24:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:22.531756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:22.532394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:22.548778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f4dc9-701c-47f7-956e-ed219443f5ef admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:22.560293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e2258-13dd-4929-ae0e-f06f4d1c1ed9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:22.561075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e2258-13dd-4929-ae0e-f06f4d1c1ed9 service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:22.561303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e2258-13dd-4929-ae0e-f06f4d1c1ed9 service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:22.561678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e2258-13dd-4929-ae0e-f06f4d1c1ed9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:22.582534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e2258-13dd-4929-ae0e-f06f4d1c1ed9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:22.584911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e2258-13dd-4929-ae0e-f06f4d1c1ed9 service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:22.612216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 885/1772] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:23.158979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.160779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.223280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:23.252150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d654-38ec-4b35-a9db-6248ded13864 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.257277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.258144 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.258424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.258698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] PATH_INFO: `/v3/projects/fc217b00af4b4528ad7dbfab9e2d55b3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.259968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc217b00af4b4528ad7dbfab9e2d55b3)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.262101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.314700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc217b00af4b4528ad7dbfab9e2d55b3', 'request_id': 'req-1b89c77e-c295-47b9-b078-776abe49e308'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.322424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc217b00af4b4528ad7dbfab9e2d55b3', 'request_id': 'req-1b89c77e-c295-47b9-b078-776abe49e308'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.328495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc217b00af4b4528ad7dbfab9e2d55b3', 'request_id': 'req-1b89c77e-c295-47b9-b078-776abe49e308'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.343182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] The token cache is being invalidate because project fc217b00af4b4528ad7dbfab9e2d55b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:23.343540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.344202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b89c77e-c295-47b9-b078-776abe49e308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.346016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 888/1773] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:23 2022] DELETE /identity/v3/projects/fc217b00af4b4528ad7dbfab9e2d55b3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:23.352665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55207-399a-4f59-b803-63ffcba39444 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.353239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55207-399a-4f59-b803-63ffcba39444 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.353239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55207-399a-4f59-b803-63ffcba39444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.374448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-44f55207-399a-4f59-b803-63ffcba39444 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:23.376053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-44f55207-399a-4f59-b803-63ffcba39444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.412705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64a1cb-36f6-4526-b19d-74f06bc7fb07 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.413205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64a1cb-36f6-4526-b19d-74f06bc7fb07 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.413288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64a1cb-36f6-4526-b19d-74f06bc7fb07 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.415223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 889/1774] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:24:23.431061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e643995-5acb-4249-a9fc-709a77268253 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.431356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e643995-5acb-4249-a9fc-709a77268253 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.431664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e643995-5acb-4249-a9fc-709a77268253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.465644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9e643995-5acb-4249-a9fc-709a77268253 None None] MFA Rules not processed for user `8e6a5d72a7da4b07ac309bf308fc4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:23.472024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9e643995-5acb-4249-a9fc-709a77268253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.482589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 886/1775] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:23.484703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c755bdb-a9ab-40ab-b04d-a774b29957e3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.485154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c755bdb-a9ab-40ab-b04d-a774b29957e3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.485598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c755bdb-a9ab-40ab-b04d-a774b29957e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.510792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7c755bdb-a9ab-40ab-b04d-a774b29957e3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:23.513155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7c755bdb-a9ab-40ab-b04d-a774b29957e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.528826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 887/1776] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:23.533655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c755bdb-a9ab-40ab-b04d-a774b29957e3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.551949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c755bdb-a9ab-40ab-b04d-a774b29957e3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.559021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.559695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.561222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.561507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.562281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:23.562591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.564627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.582132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 888/1777] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:24:23.583396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.586568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.606698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21582587-71bb-4396-9ec5-b97cd61af374 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.612119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.613000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.613288 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.613570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] PATH_INFO: `/v3/users/22edad0496ce4464882901f4733d76bd` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.614789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22edad0496ce4464882901f4733d76bd)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.617270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 890/1778] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:23.618788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e643995-5acb-4249-a9fc-709a77268253 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.622705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.630582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e643995-5acb-4249-a9fc-709a77268253 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.636655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.637341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.637627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.637904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.638803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.640896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.694770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 891/1779] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:23.694770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.696288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.699277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22edad0496ce4464882901f4733d76bd'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.701083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22edad0496ce4464882901f4733d76bd'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.717872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22edad0496ce4464882901f4733d76bd'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.720195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22edad0496ce4464882901f4733d76bd'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.750503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22edad0496ce4464882901f4733d76bd'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.757880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:23.774522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:23.776262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 889/1780] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:23 2022] DELETE /identity/v3/users/22edad0496ce4464882901f4733d76bd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:23.777457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.785022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.816053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084935f9-8bff-409e-ab12-975a4a4988b5 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.955495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.957698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.957993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.958277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.964703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb727a-80fe-4dd9-aa69-5e2605db2f64 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.964889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.983639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.984018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.984270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.984557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.985278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.987687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.006249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.006249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.022038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 892/1781] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:24.040058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 890/1782] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:24.043405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.062583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdee813-3b71-498b-b027-03ef1f2597ed service placement] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.073801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.074856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.074856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.075052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.075756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.077353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.100785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 893/1783] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:24.106429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.129912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca29d8-a2e0-4b95-a443-176d5cb38f35 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.138520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9526ab-7cd6-4391-828c-f69bcffeecc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.139271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9526ab-7cd6-4391-828c-f69bcffeecc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.139622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9526ab-7cd6-4391-828c-f69bcffeecc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.140059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9526ab-7cd6-4391-828c-f69bcffeecc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.140818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9526ab-7cd6-4391-828c-f69bcffeecc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.145298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9526ab-7cd6-4391-828c-f69bcffeecc3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.182558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 891/1784] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:24.186795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.204096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f52efc-a2c4-42f4-ae3c-66f62270de4d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.211330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.212227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.212513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.212796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] PATH_INFO: `/v3/projects/7e0413283f6d4741b0f48502409b753f/users/002bec9011a049e088a830960c4f6cd4/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.244996 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0413283f6d4741b0f48502409b753f, user_id=002bec9011a049e088a830960c4f6cd4, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.247696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.265217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 894/1785] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:24 2022] PUT /identity/v3/projects/7e0413283f6d4741b0f48502409b753f/users/002bec9011a049e088a830960c4f6cd4/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:24.278336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a6473-34c1-47f5-a8f9-c6e3b7ab5fba None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.278654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a6473-34c1-47f5-a8f9-c6e3b7ab5fba None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.279003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a6473-34c1-47f5-a8f9-c6e3b7ab5fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.318731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c29a6473-34c1-47f5-a8f9-c6e3b7ab5fba None None] MFA Rules not processed for user `002bec9011a049e088a830960c4f6cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:24.320099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c29a6473-34c1-47f5-a8f9-c6e3b7ab5fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:24.328627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.330452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:24.435954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 892/1786] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:24.436884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a6473-34c1-47f5-a8f9-c6e3b7ab5fba None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.440604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f3d67-6278-4ee7-a518-70660a516502 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.447073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.449622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.449992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.450352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] PATH_INFO: `/v3/projects/369f9c0c4b714bc982134dc9db9ba237` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.451879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] RBAC: Authorizing `identity:delete_project(project_id=369f9c0c4b714bc982134dc9db9ba237)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.454414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a6473-34c1-47f5-a8f9-c6e3b7ab5fba None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.454613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.460310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.461372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.461847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.462310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.463232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.466361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.507591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 893/1787] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:24 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:24.512171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.523472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:24.534885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369f9c0c4b714bc982134dc9db9ba237', 'request_id': 'req-a18816c7-3813-43e2-928a-04c02009864b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.549522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369f9c0c4b714bc982134dc9db9ba237', 'request_id': 'req-a18816c7-3813-43e2-928a-04c02009864b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.557756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369f9c0c4b714bc982134dc9db9ba237', 'request_id': 'req-a18816c7-3813-43e2-928a-04c02009864b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.574947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] The token cache is being invalidate because project 369f9c0c4b714bc982134dc9db9ba237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:24.575936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.576936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.578485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:24.585387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 895/1788] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:24 2022] DELETE /identity/v3/projects/369f9c0c4b714bc982134dc9db9ba237 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:24.587028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.589839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:24.600935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458415a-3246-4274-9c91-022938685d18 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.669878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.671333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.671846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.672212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.673537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.676706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.686597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18816c7-3813-43e2-928a-04c02009864b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.693262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.694605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.695136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.695612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] PATH_INFO: `/v3/users/5e17d3569e394ba180663d67a05bfe60` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.698581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e17d3569e394ba180663d67a05bfe60)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.701731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.708470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 894/1789] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:24 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:24.709711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.711914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:24.729141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015f6a2-f821-464e-949b-09a8ef4c515a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.735998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.737259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.737800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.738899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.741445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.743707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.765481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 895/1790] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:24.767276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.789140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ed37-4ded-43d9-b7b6-20b0485a5475 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.799619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.803243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.803540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.804231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] PATH_INFO: `/v3/projects/6f3b10f85dd1403db32556ea0ab60ad7/users/8ba030076a164dd1b61a0f2a52b6c30d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.807755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e17d3569e394ba180663d67a05bfe60'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.809273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e17d3569e394ba180663d67a05bfe60'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.811639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e17d3569e394ba180663d67a05bfe60'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.845984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e17d3569e394ba180663d67a05bfe60'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.847050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3b10f85dd1403db32556ea0ab60ad7, user_id=8ba030076a164dd1b61a0f2a52b6c30d, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.851391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.856807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e17d3569e394ba180663d67a05bfe60'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:24.874894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 896/1791] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:24 2022] PUT /identity/v3/projects/6f3b10f85dd1403db32556ea0ab60ad7/users/8ba030076a164dd1b61a0f2a52b6c30d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:24.880169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.882639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:24.892545 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 896/1792] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:24 2022] DELETE /identity/v3/users/5e17d3569e394ba180663d67a05bfe60 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:25.003208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639e6b-891a-4cba-bcf9-0e654c9c48c0 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.092165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:25.107076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a5936-b537-477d-8934-ed197d7eaf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.107759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a5936-b537-477d-8934-ed197d7eaf75 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.108086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a5936-b537-477d-8934-ed197d7eaf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.108358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a5936-b537-477d-8934-ed197d7eaf75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.109336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a5936-b537-477d-8934-ed197d7eaf75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.111387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a5936-b537-477d-8934-ed197d7eaf75 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.127320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 897/1793] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:25.131647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.149209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fadb5-98b8-4f77-8327-12e7aecb166b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.163242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.164121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.164408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.164684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] PATH_INFO: `/v3/projects/6f3b10f85dd1403db32556ea0ab60ad7/users/8ba030076a164dd1b61a0f2a52b6c30d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.170527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3b10f85dd1403db32556ea0ab60ad7, user_id=8ba030076a164dd1b61a0f2a52b6c30d, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.173449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.230694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 897/1794] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:25 2022] PUT /identity/v3/projects/6f3b10f85dd1403db32556ea0ab60ad7/users/8ba030076a164dd1b61a0f2a52b6c30d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:25.242247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.242890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.244238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.321114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] MFA Rules not processed for user `8ba030076a164dd1b61a0f2a52b6c30d`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:25.325586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:25.444952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 898/1795] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:25.457240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.460329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:25.544219 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.544377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:25.545997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:25.588769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e06d-cc81-4c60-9a65-01406c118237 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.604179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.611010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.611561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.612284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.667497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.670624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.690298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 898/1796] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:25.725916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bacb0-c158-4812-ae25-cfe5ccfa3757 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.739129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.739671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.740760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.741052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] PATH_INFO: `/v3/projects/7fb3a20ea17548909fa7dc28b01e3711` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.743926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb3a20ea17548909fa7dc28b01e3711)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.747124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.893251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb3a20ea17548909fa7dc28b01e3711', 'request_id': 'req-1c215c28-0c1f-48de-936d-993465a65bba'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.917258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb3a20ea17548909fa7dc28b01e3711', 'request_id': 'req-1c215c28-0c1f-48de-936d-993465a65bba'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.941527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.945873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb3a20ea17548909fa7dc28b01e3711', 'request_id': 'req-1c215c28-0c1f-48de-936d-993465a65bba'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.955534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434b76-3a78-4406-8329-e3249a606302 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.977250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] The token cache is being invalidate because project 7fb3a20ea17548909fa7dc28b01e3711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:25.977540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.978216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1c215c28-0c1f-48de-936d-993465a65bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.987970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 899/1797] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:25 2022] DELETE /identity/v3/projects/7fb3a20ea17548909fa7dc28b01e3711 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:25.990023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39f83a-69fe-4c5d-89ae-2ab836e29bad None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.990315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39f83a-69fe-4c5d-89ae-2ab836e29bad None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.990595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39f83a-69fe-4c5d-89ae-2ab836e29bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.000692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:26.032420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2f39f83a-69fe-4c5d-89ae-2ab836e29bad None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:26.033279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2f39f83a-69fe-4c5d-89ae-2ab836e29bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:26.086466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565c812-bf7a-4da5-a76c-dcd43470a030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.087183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565c812-bf7a-4da5-a76c-dcd43470a030 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.087471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565c812-bf7a-4da5-a76c-dcd43470a030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.088146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565c812-bf7a-4da5-a76c-dcd43470a030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.091039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4565c812-bf7a-4da5-a76c-dcd43470a030 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:26.203640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565c812-bf7a-4da5-a76c-dcd43470a030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.205605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565c812-bf7a-4da5-a76c-dcd43470a030 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.220406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 899/1798] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:26.222452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76326beb-ff5d-4bf8-bfa2-f26d48e09bac None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.222754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76326beb-ff5d-4bf8-bfa2-f26d48e09bac None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.223080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76326beb-ff5d-4bf8-bfa2-f26d48e09bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.231151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 900/1799] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:26.254255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-76326beb-ff5d-4bf8-bfa2-f26d48e09bac None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:26.257600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-76326beb-ff5d-4bf8-bfa2-f26d48e09bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:26.270121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 900/1800] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:26.273964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39f83a-69fe-4c5d-89ae-2ab836e29bad None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:26.289173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39f83a-69fe-4c5d-89ae-2ab836e29bad None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:26.301238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.302675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.302675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.302675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.303214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:26.303526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.305245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.341008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 901/1801] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:26.342638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76326beb-ff5d-4bf8-bfa2-f26d48e09bac None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:26.356876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76326beb-ff5d-4bf8-bfa2-f26d48e09bac None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:26.381189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.382577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.383177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.385500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.386402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.392282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.481669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 901/1802] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:26 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:26.485518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:26.500956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eedef-1efb-438d-9dca-a641852d9561 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:26.604774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.605576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.605867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.606241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.607054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.613554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.685427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 902/1803] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:26 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:26.685838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:26.708665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2461a4-0a16-4f7b-8237-ff0e62a14814 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:26.727550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.728330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.728614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.728903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.729704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.731664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.759394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:26.764179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 902/1804] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:26.783576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5cf5a-6bda-4a18-9dfc-018920675367 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:26.791580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4090d-8f1f-4e84-9a39-c88194c149a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.793829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4090d-8f1f-4e84-9a39-c88194c149a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.794274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4090d-8f1f-4e84-9a39-c88194c149a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.794874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4090d-8f1f-4e84-9a39-c88194c149a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.796885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4090d-8f1f-4e84-9a39-c88194c149a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.800711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4090d-8f1f-4e84-9a39-c88194c149a5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.847674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 903/1805] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:26.848170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:26.868911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9b24-06a1-43e2-9964-1d581aea01ff admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:26.880603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.881511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.881792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.882066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] PATH_INFO: `/v3/projects/234f88d75861439c81fc82d18d1532c5/users/7603bc8e01b04a0c89ed2bde93fdb006/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.921428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234f88d75861439c81fc82d18d1532c5, user_id=7603bc8e01b04a0c89ed2bde93fdb006, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.924971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.969986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 903/1806] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:26 2022] PUT /identity/v3/projects/234f88d75861439c81fc82d18d1532c5/users/7603bc8e01b04a0c89ed2bde93fdb006/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:26.974498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e4661-a6f2-43e3-8402-3a9df5fe814b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.975449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e4661-a6f2-43e3-8402-3a9df5fe814b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.976523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e4661-a6f2-43e3-8402-3a9df5fe814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.019110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a13e4661-a6f2-43e3-8402-3a9df5fe814b None None] MFA Rules not processed for user `7603bc8e01b04a0c89ed2bde93fdb006`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:27.021027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a13e4661-a6f2-43e3-8402-3a9df5fe814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:27.112233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:27.127757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.130681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 904/1807] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:27.170472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:27.232688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf3091-e04b-49f6-8f92-28dbeb57eb03 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.237909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.238743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.239094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.239381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.240148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.243724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.286641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 904/1808] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:27 2022] POST /identity/v3/projects => generated 369 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:27.291598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e4661-a6f2-43e3-8402-3a9df5fe814b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.304987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e4661-a6f2-43e3-8402-3a9df5fe814b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.445908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.446909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.448205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.448729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.450666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.454270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.502336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 905/1809] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:27 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:27.506271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.525254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa5370-f92f-4515-a2f1-ef15f7dd72a8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.530932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.531626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.531947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.532436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.533185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.535483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.549849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 905/1810] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:27.555334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.583954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca642c3-900d-47fb-bc9c-0c21c1956d56 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.599675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.600983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.602136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.602136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] PATH_INFO: `/v3/projects/a5438cb2e8a942a2b9daf42d4a4735fd/users/042f57ec9bc54b3c9ad082d363271263/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.641560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5438cb2e8a942a2b9daf42d4a4735fd, user_id=042f57ec9bc54b3c9ad082d363271263, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.652278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.685948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.686854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 906/1811] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:27 2022] PUT /identity/v3/projects/a5438cb2e8a942a2b9daf42d4a4735fd/users/042f57ec9bc54b3c9ad082d363271263/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:27.705069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79b0f-c12e-4175-b18c-b446debd49d8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.884530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.897270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:27.925203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.926280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.926826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.927302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.928320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.930427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.939547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2108e-5a54-4efa-9098-c0fc4a7728c5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.960872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 906/1812] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:27.964095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.969748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.970919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.971374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.971803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] PATH_INFO: `/v3/users/8cda70c9df71418fb92860f03f4e6fda` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.975186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cda70c9df71418fb92860f03f4e6fda)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.977374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.006932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e894-08b0-48af-a681-597ed20137e4 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.024582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de3395-31f6-45cc-9cf7-b3d83447392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.038262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3395-31f6-45cc-9cf7-b3d83447392c admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.038563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3395-31f6-45cc-9cf7-b3d83447392c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.040893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3395-31f6-45cc-9cf7-b3d83447392c admin admin] PATH_INFO: `/v3/projects/a5438cb2e8a942a2b9daf42d4a4735fd/users/042f57ec9bc54b3c9ad082d363271263/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.046908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de3395-31f6-45cc-9cf7-b3d83447392c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5438cb2e8a942a2b9daf42d4a4735fd, user_id=042f57ec9bc54b3c9ad082d363271263, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.049649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de3395-31f6-45cc-9cf7-b3d83447392c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.073470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 907/1813] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:27 2022] PUT /identity/v3/projects/a5438cb2e8a942a2b9daf42d4a4735fd/users/042f57ec9bc54b3c9ad082d363271263/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:28.090642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.091042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.092895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.162111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cda70c9df71418fb92860f03f4e6fda'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.169325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cda70c9df71418fb92860f03f4e6fda'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.200247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cda70c9df71418fb92860f03f4e6fda'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.200938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cda70c9df71418fb92860f03f4e6fda'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.223421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-282ee7c3-1d98-4004-9815-f76c47390ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cda70c9df71418fb92860f03f4e6fda'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.235027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] MFA Rules not processed for user `042f57ec9bc54b3c9ad082d363271263`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:28.240004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.250500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 907/1814] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:27 2022] DELETE /identity/v3/users/8cda70c9df71418fb92860f03f4e6fda => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:28.257316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.257621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.257907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.318438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:28.320186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.445710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:28.456081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 908/1815] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:28.491504 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.497416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.500509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 908/1816] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:28.525589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.527780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.557956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:28.740147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbeab0-e977-46b3-997e-bd974d59fa2e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.780513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376ea55-0b8b-483e-9dc6-72770e24d6fe None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.788702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8669ce8-4808-4a1f-89e2-940aeb9cb802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.789499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8669ce8-4808-4a1f-89e2-940aeb9cb802 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.789944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8669ce8-4808-4a1f-89e2-940aeb9cb802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.790542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8669ce8-4808-4a1f-89e2-940aeb9cb802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.834118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8669ce8-4808-4a1f-89e2-940aeb9cb802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.836588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8669ce8-4808-4a1f-89e2-940aeb9cb802 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.852886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 909/1817] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:28.855228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8a9f-22cb-4769-ac06-373ec9b24f63 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.855528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8a9f-22cb-4769-ac06-373ec9b24f63 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.855814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8a9f-22cb-4769-ac06-373ec9b24f63 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.857817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 910/1818] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:24:28.879511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ce282-60d1-49f5-bcad-cdf2a6b415db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.880204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ce282-60d1-49f5-bcad-cdf2a6b415db service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.880498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ce282-60d1-49f5-bcad-cdf2a6b415db service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.880782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ce282-60d1-49f5-bcad-cdf2a6b415db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.892357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab4cb8-e8d3-49cc-9d41-31b66f79e2a6 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.892588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab4cb8-e8d3-49cc-9d41-31b66f79e2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.892904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab4cb8-e8d3-49cc-9d41-31b66f79e2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.951125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b3ab4cb8-e8d3-49cc-9d41-31b66f79e2a6 None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:28.959472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b3ab4cb8-e8d3-49cc-9d41-31b66f79e2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.039647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ce282-60d1-49f5-bcad-cdf2a6b415db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:29.041190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ce282-60d1-49f5-bcad-cdf2a6b415db service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:29.066622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 909/1819] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:29.129017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 911/1820] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:29.139766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a280a-648d-49c0-99c3-711eddf9554d None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.140494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a280a-648d-49c0-99c3-711eddf9554d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.140845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a280a-648d-49c0-99c3-711eddf9554d None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:29.142707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 910/1821] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:24:29.154539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.154834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.155113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:29.198454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:29.204964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.235911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 912/1822] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:24:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:29.376526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a280a-648d-49c0-99c3-711eddf9554d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:29.379129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-387a280a-648d-49c0-99c3-711eddf9554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.413589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a280a-648d-49c0-99c3-711eddf9554d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:29.423531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:29.423989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.424760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.424760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:29.439276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:29.440941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:29.456827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 911/1823] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:29.478473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:29.480520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.516468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebd6a6-b1b3-409a-bda7-22cc7144b958 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:29.533502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3494969-ed1d-45ec-925d-21b0abe0b34e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:29.535801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3494969-ed1d-45ec-925d-21b0abe0b34e service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.536223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3494969-ed1d-45ec-925d-21b0abe0b34e service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.537211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3494969-ed1d-45ec-925d-21b0abe0b34e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:29.560580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3494969-ed1d-45ec-925d-21b0abe0b34e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:29.562333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3494969-ed1d-45ec-925d-21b0abe0b34e service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:29.597201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 913/1824] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:29.696841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:29.705195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.749818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf52e56-7a27-4fd4-bae8-b5fbf46070ba service nova] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:29.756248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:29.758013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.759289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.760148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] PATH_INFO: `/v3/users/002bec9011a049e088a830960c4f6cd4` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:29.762426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002bec9011a049e088a830960c4f6cd4)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:29.765681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:29.888599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002bec9011a049e088a830960c4f6cd4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:29.890155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002bec9011a049e088a830960c4f6cd4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:29.891735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002bec9011a049e088a830960c4f6cd4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:29.919349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:29.933794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002bec9011a049e088a830960c4f6cd4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:29.948120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002bec9011a049e088a830960c4f6cd4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:29.981481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 912/1825] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:29 2022] DELETE /identity/v3/users/002bec9011a049e088a830960c4f6cd4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:29.983460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.983929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.984337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:30.036094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:30.037339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:30.190771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:30.222811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 914/1826] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:30.233818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:30.235560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:30.264693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:30.269088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:30.289292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d25a8-4a26-4438-8e0b-287c38d6e4ac None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:30.305747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:30.308205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:30.308943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:30.309498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] PATH_INFO: `/v3/projects/3b0ca4ed871d46be9bb293eae9f8b5f9` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:30.312137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0ca4ed871d46be9bb293eae9f8b5f9)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:30.314590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:30.343188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61bf8a-48d9-4d5a-a043-c36bf13fe605 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:30.351534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:30.352406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:30.352846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:30.353260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:30.371981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:30.376892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:30.395406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 913/1827] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:30.474057 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0ca4ed871d46be9bb293eae9f8b5f9', 'request_id': 'req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.487724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0ca4ed871d46be9bb293eae9f8b5f9', 'request_id': 'req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.506588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0ca4ed871d46be9bb293eae9f8b5f9', 'request_id': 'req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.532574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] The token cache is being invalidate because project 3b0ca4ed871d46be9bb293eae9f8b5f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:30.532923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.534793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.538640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 915/1828] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:30 2022] DELETE /identity/v3/projects/3b0ca4ed871d46be9bb293eae9f8b5f9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:30.543258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:30.554710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7408de-c5ba-4b62-8773-3a0fe7394e10 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:30.768561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:30.769574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:30.769860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:30.770138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] PATH_INFO: `/v3/users/2d834a989ba84db9880463500fec06ec` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:30.772539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d834a989ba84db9880463500fec06ec)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:30.787287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:30.912552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d834a989ba84db9880463500fec06ec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.913572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d834a989ba84db9880463500fec06ec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.914347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d834a989ba84db9880463500fec06ec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.970052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d834a989ba84db9880463500fec06ec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:30.985897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d834a989ba84db9880463500fec06ec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.003739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:31.014304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 914/1829] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:30 2022] DELETE /identity/v3/users/2d834a989ba84db9880463500fec06ec => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:31.206832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:31.208713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:31.241989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01888c-ce9e-47a4-be39-a59b4ebd59d8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:31.390169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:31.405058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:31.405738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:31.406655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] PATH_INFO: `/v3/projects/7e0413283f6d4741b0f48502409b753f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:31.409071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0413283f6d4741b0f48502409b753f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:31.413485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:31.531618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0413283f6d4741b0f48502409b753f', 'request_id': 'req-036df151-bdef-4234-9e19-e9b69a8a9bea'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.544362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0413283f6d4741b0f48502409b753f', 'request_id': 'req-036df151-bdef-4234-9e19-e9b69a8a9bea'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.567461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0413283f6d4741b0f48502409b753f', 'request_id': 'req-036df151-bdef-4234-9e19-e9b69a8a9bea'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.586977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] The token cache is being invalidate because project 7e0413283f6d4741b0f48502409b753f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:31.587269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.588042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.592831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 916/1830] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:31 2022] DELETE /identity/v3/projects/7e0413283f6d4741b0f48502409b753f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:31.598068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:31.601024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:31.616229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23076bbf-2e08-4de3-b7bf-9927b31cc8ab admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:31.620226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:31.621740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:31.631410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:31.708023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:31.709450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:31.709886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:31.710346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] PATH_INFO: `/v3/users/8ba030076a164dd1b61a0f2a52b6c30d` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:31.712082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba030076a164dd1b61a0f2a52b6c30d)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:31.714690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:31.722717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df151-bdef-4234-9e19-e9b69a8a9bea admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:31.729627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:31.730688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:31.731078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:31.731585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] PATH_INFO: `/v3/users/7603bc8e01b04a0c89ed2bde93fdb006` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:31.733298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7603bc8e01b04a0c89ed2bde93fdb006)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:31.735951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:31.791161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba030076a164dd1b61a0f2a52b6c30d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.793396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba030076a164dd1b61a0f2a52b6c30d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.794875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba030076a164dd1b61a0f2a52b6c30d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.808235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7603bc8e01b04a0c89ed2bde93fdb006'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.809703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7603bc8e01b04a0c89ed2bde93fdb006'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.816766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba030076a164dd1b61a0f2a52b6c30d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.824818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7603bc8e01b04a0c89ed2bde93fdb006'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.825952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7603bc8e01b04a0c89ed2bde93fdb006'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.828180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5c24e290-f05e-4c18-93c2-2c8111477061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba030076a164dd1b61a0f2a52b6c30d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.848077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7603bc8e01b04a0c89ed2bde93fdb006'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:31.849597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 915/1831] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:31 2022] DELETE /identity/v3/users/8ba030076a164dd1b61a0f2a52b6c30d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:31.864502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 917/1832] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:31 2022] DELETE /identity/v3/users/7603bc8e01b04a0c89ed2bde93fdb006 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:31.870097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:31.870411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:31.870652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:31.890186 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:31.891390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:31.997672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 916/1833] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:32.008602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:32.011175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:32.087734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d45be-c2f3-44a5-830e-1c54bf901a4a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:32.093195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:32.094071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.094426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.094782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.109753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:32.111761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:32.129529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 918/1834] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:32.382931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:32.384574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:32.397540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b374a32-5a9a-4b3a-a1ad-39c4e8ff880f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:32.403647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:32.404699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.405050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.405644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] PATH_INFO: `/v3/projects/4e4e51e779dd4d628573d3a22a5b693f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.408251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4e51e779dd4d628573d3a22a5b693f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:32.410266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:32.425638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:32.462048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:32.464295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:32.495300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4e51e779dd4d628573d3a22a5b693f', 'request_id': 'req-7fde90b4-b876-43c6-b3bc-f405a1e8360f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.510352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdd3f4-af80-4919-b7bf-44e744a4abbd service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:32.513819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4e51e779dd4d628573d3a22a5b693f', 'request_id': 'req-7fde90b4-b876-43c6-b3bc-f405a1e8360f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.528593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4e51e779dd4d628573d3a22a5b693f', 'request_id': 'req-7fde90b4-b876-43c6-b3bc-f405a1e8360f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.545761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] The token cache is being invalidate because project 4e4e51e779dd4d628573d3a22a5b693f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:32.546151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.546885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7fde90b4-b876-43c6-b3bc-f405a1e8360f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.552109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 917/1835] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:32 2022] DELETE /identity/v3/projects/4e4e51e779dd4d628573d3a22a5b693f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:32.553719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:32.555874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612df0-f40e-4645-a3a4-11107ad1edd6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.556221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612df0-f40e-4645-a3a4-11107ad1edd6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.556570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612df0-f40e-4645-a3a4-11107ad1edd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.592828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-77612df0-f40e-4645-a3a4-11107ad1edd6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:32.594766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-77612df0-f40e-4645-a3a4-11107ad1edd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:32.611421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:32.612489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.612982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.613342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] PATH_INFO: `/v3/projects/234f88d75861439c81fc82d18d1532c5` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.614808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234f88d75861439c81fc82d18d1532c5)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:32.617693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:32.692999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 918/1836] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:32.701037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102ed42-3714-492e-a07f-94e23c15b503 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.701476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102ed42-3714-492e-a07f-94e23c15b503 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.702388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102ed42-3714-492e-a07f-94e23c15b503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.738082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234f88d75861439c81fc82d18d1532c5', 'request_id': 'req-295f9857-13ba-4e1d-a909-f14c3dfed895'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.740835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d102ed42-3714-492e-a07f-94e23c15b503 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:32.742083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d102ed42-3714-492e-a07f-94e23c15b503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:32.748615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234f88d75861439c81fc82d18d1532c5', 'request_id': 'req-295f9857-13ba-4e1d-a909-f14c3dfed895'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.762699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234f88d75861439c81fc82d18d1532c5', 'request_id': 'req-295f9857-13ba-4e1d-a909-f14c3dfed895'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.789999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] The token cache is being invalidate because project 234f88d75861439c81fc82d18d1532c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:32.790484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.791765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:32.795100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 919/1837] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:32 2022] DELETE /identity/v3/projects/234f88d75861439c81fc82d18d1532c5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:32.800013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:32.804325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:32.833057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 919/1838] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:32.839674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102ed42-3714-492e-a07f-94e23c15b503 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:32.851989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102ed42-3714-492e-a07f-94e23c15b503 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:32.858395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:32.859901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.860345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.861214 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.862159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:32.862634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:32.864739 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:32.892541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 920/1839] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:32.895158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:32.921752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cf90a-6a61-4b9b-811b-51576bb684f0 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:32.933485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:32.934171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.934459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.934737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.935450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:32.941956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:32.943086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f9857-13ba-4e1d-a909-f14c3dfed895 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:32.956276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:32.958198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:32.958198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:32.958198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] PATH_INFO: `/v3/users/042f57ec9bc54b3c9ad082d363271263` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:32.964558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=042f57ec9bc54b3c9ad082d363271263)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:32.967028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:33.017259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 921/1840] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:32 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:33.018016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:33.033970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:33.070357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042f57ec9bc54b3c9ad082d363271263'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.075507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042f57ec9bc54b3c9ad082d363271263'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.095569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042f57ec9bc54b3c9ad082d363271263'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.096381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042f57ec9bc54b3c9ad082d363271263'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.129159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:33.148965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042f57ec9bc54b3c9ad082d363271263'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.184342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 920/1841] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:32 2022] DELETE /identity/v3/users/042f57ec9bc54b3c9ad082d363271263 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:33.209217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdd477-45ad-4d10-8336-1ac932b4c1ea admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:33.317571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:33.318540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.318969 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.319799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.320732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:33.322991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:33.382091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 922/1842] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:33 2022] POST /identity/v3/users => generated 293 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:33.388108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:33.390693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:33.418397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b0158-b4c8-4894-99ac-2800024deb4d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:33.428684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:33.430539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:33.432841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:33.434748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.435297 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.437410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.438193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:33.451862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:33.457268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796503-54db-417d-aaff-9a07ecacc9b3 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:33.469051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 921/1843] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:33.473169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:33.476864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:33.477879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.478316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.479312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] PATH_INFO: `/v3/projects/6f3b10f85dd1403db32556ea0ab60ad7` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.482770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3b10f85dd1403db32556ea0ab60ad7)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:33.485427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:33.491438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b140b-92f3-4db1-b42e-1dee0b35d3f4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:33.503311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:33.504167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.504606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.504994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.506016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:33.526848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:33.568312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 922/1844] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:33.569278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:33.589982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e6edf-4098-4b01-bdb6-06faf1be5cb4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:33.596366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3b10f85dd1403db32556ea0ab60ad7', 'request_id': 'req-129bd42c-0cd5-49ac-b454-d68fd6796a67'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.605375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3b10f85dd1403db32556ea0ab60ad7', 'request_id': 'req-129bd42c-0cd5-49ac-b454-d68fd6796a67'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.614210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3b10f85dd1403db32556ea0ab60ad7', 'request_id': 'req-129bd42c-0cd5-49ac-b454-d68fd6796a67'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.628672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] The token cache is being invalidate because project 6f3b10f85dd1403db32556ea0ab60ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:33.628894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.629647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-129bd42c-0cd5-49ac-b454-d68fd6796a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:33.631982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 923/1845] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:33 2022] DELETE /identity/v3/projects/6f3b10f85dd1403db32556ea0ab60ad7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:33.639450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eac21-081f-4412-bc11-ebe18a60ac7a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.639745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eac21-081f-4412-bc11-ebe18a60ac7a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.641779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eac21-081f-4412-bc11-ebe18a60ac7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.663004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c16eac21-081f-4412-bc11-ebe18a60ac7a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:33.664341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c16eac21-081f-4412-bc11-ebe18a60ac7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:33.694171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:33.733071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183f5d7-01de-4d0b-a025-4ee96ab701e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:33.734263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f5d7-01de-4d0b-a025-4ee96ab701e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.734657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f5d7-01de-4d0b-a025-4ee96ab701e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.735153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f5d7-01de-4d0b-a025-4ee96ab701e0 admin admin] PATH_INFO: `/v3/projects/7a7e5de723834100881245e603802e96/users/c7a4360e99e04c4a8630bda3b61a9267/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.793557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 924/1846] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:33.802192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183f5d7-01de-4d0b-a025-4ee96ab701e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7e5de723834100881245e603802e96, user_id=c7a4360e99e04c4a8630bda3b61a9267, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:33.805100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.805711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.806146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.810056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183f5d7-01de-4d0b-a025-4ee96ab701e0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:33.846017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 923/1847] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:33 2022] PUT /identity/v3/projects/7a7e5de723834100881245e603802e96/users/c7a4360e99e04c4a8630bda3b61a9267/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:33.850200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cbf7d-fdcf-4398-88fb-5353fb82acad None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:33.850590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cbf7d-fdcf-4398-88fb-5353fb82acad None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:33.851012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cbf7d-fdcf-4398-88fb-5353fb82acad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:33.856875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:33.860301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:33.905271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-253cbf7d-fdcf-4398-88fb-5353fb82acad None None] MFA Rules not processed for user `c7a4360e99e04c4a8630bda3b61a9267`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:33.906050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-253cbf7d-fdcf-4398-88fb-5353fb82acad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:33.984805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 925/1848] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:33.993090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:33.997628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:34.035548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 924/1849] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:34.036585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cbf7d-fdcf-4398-88fb-5353fb82acad None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.050828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5ff0a-2e64-4e43-8963-4c8297394ee7 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.060876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.061700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cbf7d-fdcf-4398-88fb-5353fb82acad None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.061826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.063032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.063303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] PATH_INFO: `/v3/projects/a5438cb2e8a942a2b9daf42d4a4735fd` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.065514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5438cb2e8a942a2b9daf42d4a4735fd)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.067513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.075465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.076277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.076632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.076984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.078275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:34.078275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.085138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.116846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 925/1850] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:34.120820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.127015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:34.136553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5438cb2e8a942a2b9daf42d4a4735fd', 'request_id': 'req-fe9ef258-8f8a-49b2-b546-6ae5811a1408'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:34.142766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319a685-070b-40f6-81b2-7b7335ee1c8a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.147442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5438cb2e8a942a2b9daf42d4a4735fd', 'request_id': 'req-fe9ef258-8f8a-49b2-b546-6ae5811a1408'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:34.153633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:34.160722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5438cb2e8a942a2b9daf42d4a4735fd', 'request_id': 'req-fe9ef258-8f8a-49b2-b546-6ae5811a1408'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:34.176842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] The token cache is being invalidate because project a5438cb2e8a942a2b9daf42d4a4735fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:34.178348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:34.179225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:34.184433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 926/1851] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:33 2022] DELETE /identity/v3/projects/a5438cb2e8a942a2b9daf42d4a4735fd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:34.184433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.184748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:34.258126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854960d-feb1-44cd-9899-ee8f86378bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.259468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854960d-feb1-44cd-9899-ee8f86378bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.260615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854960d-feb1-44cd-9899-ee8f86378bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.261412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:34.261662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854960d-feb1-44cd-9899-ee8f86378bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.263766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854960d-feb1-44cd-9899-ee8f86378bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.267871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854960d-feb1-44cd-9899-ee8f86378bdf admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.292659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef258-8f8a-49b2-b546-6ae5811a1408 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.306223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.307791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.308264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.308686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.309538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.311609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.313688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 926/1852] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:34.318959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94390cf6-559b-4962-8906-75bc222e56ea None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.319703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94390cf6-559b-4962-8906-75bc222e56ea None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.320038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94390cf6-559b-4962-8906-75bc222e56ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.346051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-94390cf6-559b-4962-8906-75bc222e56ea None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:34.347292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-94390cf6-559b-4962-8906-75bc222e56ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:34.358885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 927/1853] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:34.359765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.361619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:34.438128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5587-5ff1-4ff8-8d2e-1d7c15055044 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.440470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 927/1854] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:34.441401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94390cf6-559b-4962-8906-75bc222e56ea None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.452104 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f31e8e-d31c-46ea-8dda-ea19fe4bef7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.452886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31e8e-d31c-46ea-8dda-ea19fe4bef7a admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.453292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31e8e-d31c-46ea-8dda-ea19fe4bef7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.453971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31e8e-d31c-46ea-8dda-ea19fe4bef7a admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.454821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f31e8e-d31c-46ea-8dda-ea19fe4bef7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.456808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94390cf6-559b-4962-8906-75bc222e56ea None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.459089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f31e8e-d31c-46ea-8dda-ea19fe4bef7a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.463988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.464684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.464966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.465238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.465943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.467993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.487143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 928/1855] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:34.488661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e165cf-65ca-4901-8068-f3e5606798b0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.488917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e165cf-65ca-4901-8068-f3e5606798b0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.489202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e165cf-65ca-4901-8068-f3e5606798b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.498753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 928/1856] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/users => generated 292 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:34.500095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.508908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c8e165cf-65ca-4901-8068-f3e5606798b0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:34.510897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c8e165cf-65ca-4901-8068-f3e5606798b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:34.513233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b08732-e88a-4c80-8097-4b7515ac5443 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.518786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.519562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 929/1857] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:34.520182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.520415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e165cf-65ca-4901-8068-f3e5606798b0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.521012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.521394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.522278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.524046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.540769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 929/1858] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:34.540769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.541117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e165cf-65ca-4901-8068-f3e5606798b0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.546477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.547221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.547507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.547927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.548595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.550367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.551418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40c87-fabd-4e39-9226-a00ef98ef642 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.557436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.558186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.558568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.558917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.560002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:34.560388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.562200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.563878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 930/1859] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:34.564813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.575978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655f2b8-8b98-4cfd-aec9-cb85cc3d1eda admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.583905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 930/1860] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:34.584387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.584500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.584611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.584611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.584611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] PATH_INFO: `/v3/projects/52d9019a270f42879deb54e79fda3abc/users/408163a245c3441fa4b432d4257ee5fb/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.601483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d9019a270f42879deb54e79fda3abc, user_id=408163a245c3441fa4b432d4257ee5fb, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.605313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.607510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e60fc6-0963-4363-8859-82c2c19afe3d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.624237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.624946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.625210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.625485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.626293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.628908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.647735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 931/1861] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:34 2022] PUT /identity/v3/projects/52d9019a270f42879deb54e79fda3abc/users/408163a245c3441fa4b432d4257ee5fb/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:34.649207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.654321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 931/1862] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:34.655038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.691590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cec53-4644-4beb-84be-6599a6cf1f4e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.741141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd6cf9-a26c-4cd7-91dd-be3549641530 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.751784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.752680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.752796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.753028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.753736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.755663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.764239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:34.785714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.787257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.787257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.787523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.788328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.790988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.797885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 932/1863] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:34.799158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.813869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:34.815229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 932/1864] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:34.816247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.833447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea43850-f76c-4f39-8fbe-5d92f79c0493 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.896637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93ba2d-a231-4257-b97d-dca2797da5f9 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.899008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.899693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.900007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.900317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.901044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.903103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.903273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:34.904403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:34.904827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:34.905216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] PATH_INFO: `/v3/projects/6b9bfc3dc4ba47aeb34c60d5b5baf9d9/users/ebef406a97ad48a18202b4f769e9d39c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:34.928146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b9bfc3dc4ba47aeb34c60d5b5baf9d9, user_id=ebef406a97ad48a18202b4f769e9d39c, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:34.933434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:34.939010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 933/1865] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:34 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:34.939872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:34.952666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010aa2a-0b01-48c6-9e57-965d1763d0f4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:34.959658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 933/1866] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:34 2022] PUT /identity/v3/projects/6b9bfc3dc4ba47aeb34c60d5b5baf9d9/users/ebef406a97ad48a18202b4f769e9d39c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:34.963995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:35.054790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a5efa-c9b5-469c-a270-b02984fde240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:35.055623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a5efa-c9b5-469c-a270-b02984fde240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.055936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a5efa-c9b5-469c-a270-b02984fde240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.056356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a5efa-c9b5-469c-a270-b02984fde240 admin admin] PATH_INFO: `/v3/projects/52d9019a270f42879deb54e79fda3abc/users/408163a245c3441fa4b432d4257ee5fb/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.059638 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a5efa-c9b5-469c-a270-b02984fde240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d9019a270f42879deb54e79fda3abc, user_id=408163a245c3441fa4b432d4257ee5fb, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:35.069074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a5efa-c9b5-469c-a270-b02984fde240 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:35.074835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b8bd4-e153-45c0-8a55-31724e523024 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:35.081431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24817988-bff8-42a7-87c2-fbd831d80a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:35.082237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24817988-bff8-42a7-87c2-fbd831d80a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.082618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24817988-bff8-42a7-87c2-fbd831d80a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.083022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24817988-bff8-42a7-87c2-fbd831d80a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.084280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24817988-bff8-42a7-87c2-fbd831d80a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:35.086924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24817988-bff8-42a7-87c2-fbd831d80a64 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:35.089831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 934/1867] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:34 2022] PUT /identity/v3/projects/52d9019a270f42879deb54e79fda3abc/users/408163a245c3441fa4b432d4257ee5fb/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:35.090921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f476102-2181-45b5-a528-67d6b07fffca None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.091210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f476102-2181-45b5-a528-67d6b07fffca None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.091489 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f476102-2181-45b5-a528-67d6b07fffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.124801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 934/1868] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:35.129488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2270a-b039-4575-ad88-68e4b98ed906 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.142799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2270a-b039-4575-ad88-68e4b98ed906 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.143476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2270a-b039-4575-ad88-68e4b98ed906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.179323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-1f476102-2181-45b5-a528-67d6b07fffca None None] MFA Rules not processed for user `ebef406a97ad48a18202b4f769e9d39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:35.180580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1f476102-2181-45b5-a528-67d6b07fffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:35.188620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1ef2270a-b039-4575-ad88-68e4b98ed906 None None] MFA Rules not processed for user `408163a245c3441fa4b432d4257ee5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:35.190306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1ef2270a-b039-4575-ad88-68e4b98ed906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:35.293672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:35.311476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 935/1869] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:35.311476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f476102-2181-45b5-a528-67d6b07fffca None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:35.324781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 935/1870] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:35.325807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2270a-b039-4575-ad88-68e4b98ed906 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:35.359560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:35.481217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f476102-2181-45b5-a528-67d6b07fffca None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:35.508005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:35.508710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.508996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.509276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.510582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:35.512245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:35.528075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2270a-b039-4575-ad88-68e4b98ed906 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:35.540357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:35.542507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.543427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.544259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.547266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:35.554532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 936/1871] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:35.554532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:35.559911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:35.560060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:35.622316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 936/1872] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:35 2022] POST /identity/v3/projects => generated 383 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:35.624009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:35.717906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210450-f1f7-45b6-ab7e-c9b4ff7451e5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:35.795242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe71684-b9c0-4781-b513-03f3eca56fbe admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:35.807909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:35.809028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.809262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.809543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] PATH_INFO: `/v3/projects/50d2c30221cc419495a1a83f01858025/users/589746874d8a43fa84541d354033b720/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.832244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:35.833232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.834723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.835002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.843895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d2c30221cc419495a1a83f01858025, user_id=589746874d8a43fa84541d354033b720, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:35.847733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:35.875866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 937/1873] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:35 2022] PUT /identity/v3/projects/50d2c30221cc419495a1a83f01858025/users/589746874d8a43fa84541d354033b720/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:35.876486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:35.969798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d9f0d-3e23-4643-89a4-5215a37b50cc admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:35.975626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f7a21-f707-46d3-81f1-d59e2efaf1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:35.976461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7a21-f707-46d3-81f1-d59e2efaf1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:35.976774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7a21-f707-46d3-81f1-d59e2efaf1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:35.977078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7a21-f707-46d3-81f1-d59e2efaf1e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:35.977788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f7a21-f707-46d3-81f1-d59e2efaf1e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:35.979880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f7a21-f707-46d3-81f1-d59e2efaf1e0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:35.984600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:35.986466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:35.994697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:36.003225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 938/1874] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:35 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:36.005068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43e4c0-8297-45e9-b620-51905c5748df None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.005394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43e4c0-8297-45e9-b620-51905c5748df None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.005652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 937/1875] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:36.005749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43e4c0-8297-45e9-b620-51905c5748df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.008691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.025142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da2a11-6b63-449d-aaee-b7b654192375 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.031862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.032985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.033384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.034654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.041885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.041885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.042080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:36.052183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4e43e4c0-8297-45e9-b620-51905c5748df None None] MFA Rules not processed for user `589746874d8a43fa84541d354033b720`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:36.054088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4e43e4c0-8297-45e9-b620-51905c5748df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.055507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 938/1876] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:36.060221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.074143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d97c-812e-4e43-bf97-66d0e0cb1a70 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.082525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.083842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.084538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.084917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] PATH_INFO: `/v3/projects/1d96d8b4813141d0bf8561570d6c291b/users/b60210cce9bf4b2b8119d31e8700e3dc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.118874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d96d8b4813141d0bf8561570d6c291b, user_id=b60210cce9bf4b2b8119d31e8700e3dc, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.123010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.155304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 939/1878] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:36 2022] PUT /identity/v3/projects/1d96d8b4813141d0bf8561570d6c291b/users/b60210cce9bf4b2b8119d31e8700e3dc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:36.155500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 939/1878] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:36.155597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.156504 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43e4c0-8297-45e9-b620-51905c5748df None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.183111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43e4c0-8297-45e9-b620-51905c5748df None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.253773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba3a9e-63eb-47b4-9c7f-83097d4832e5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.261250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.262392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.262677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.262979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] PATH_INFO: `/v3/users/c7a4360e99e04c4a8630bda3b61a9267` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.264697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a4360e99e04c4a8630bda3b61a9267)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.267011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.270471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.271443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.271716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.272080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.273018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.276138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.309784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 940/1879] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:36 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:36.310744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.342279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a4360e99e04c4a8630bda3b61a9267'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:36.342279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a4360e99e04c4a8630bda3b61a9267'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:36.346504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a4360e99e04c4a8630bda3b61a9267'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:36.347497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a4360e99e04c4a8630bda3b61a9267'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:36.360091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a4360e99e04c4a8630bda3b61a9267'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:36.373068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 940/1880] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:36 2022] DELETE /identity/v3/users/c7a4360e99e04c4a8630bda3b61a9267 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:36.373954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.377003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.379247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f3e52-1fa9-4158-8a61-1e9c3c471022 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.408380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:36.424690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:36.439051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7234cd-c842-4293-b87b-c634ea3acd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.439730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7234cd-c842-4293-b87b-c634ea3acd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.440061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7234cd-c842-4293-b87b-c634ea3acd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.440378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7234cd-c842-4293-b87b-c634ea3acd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.441147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7234cd-c842-4293-b87b-c634ea3acd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.442751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7234cd-c842-4293-b87b-c634ea3acd0e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.446196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e62c57-da67-4732-b2bb-a9d24158e15f admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.451194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.451891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.452176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.452451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.452696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 941/1881] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:36.453195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.453918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263cda-3e3a-4de7-a467-58f2a03607c0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.454205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263cda-3e3a-4de7-a467-58f2a03607c0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.454484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263cda-3e3a-4de7-a467-58f2a03607c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.455134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.472578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-78263cda-3e3a-4de7-a467-58f2a03607c0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:36.473771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-78263cda-3e3a-4de7-a467-58f2a03607c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.476132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 941/1882] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:36 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:36.476927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.479247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.482195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 942/1883] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:36.482985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263cda-3e3a-4de7-a467-58f2a03607c0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.491800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66d3ef-769e-4fef-a121-6087d73bbcd6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.492053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263cda-3e3a-4de7-a467-58f2a03607c0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.497168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.497324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.497982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.498090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.498275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.498404 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.498558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.498662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] PATH_INFO: `/v3/projects/1d96d8b4813141d0bf8561570d6c291b/users/b60210cce9bf4b2b8119d31e8700e3dc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.499296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.500713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d96d8b4813141d0bf8561570d6c291b, user_id=b60210cce9bf4b2b8119d31e8700e3dc, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.500949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.503395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.510868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 943/1884] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:36.511650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.515255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.516066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 942/1885] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:36 2022] PUT /identity/v3/projects/1d96d8b4813141d0bf8561570d6c291b/users/b60210cce9bf4b2b8119d31e8700e3dc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:36.524729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.572087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc5d97-bcc5-4cba-aafe-8154582e5695 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.588314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360711af-893b-4f07-83b7-9a492167f778 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.594644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2396-844f-489d-9b9f-f9ca1eac978b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.595612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2396-844f-489d-9b9f-f9ca1eac978b admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.596109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2396-844f-489d-9b9f-f9ca1eac978b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.596492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2396-844f-489d-9b9f-f9ca1eac978b admin admin] PATH_INFO: `/v3/projects/7b1e99faded24e9f83c11402ba92358f/users/cf9ffbc895324c999d179eab327e1949/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.614556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d2396-844f-489d-9b9f-f9ca1eac978b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1e99faded24e9f83c11402ba92358f, user_id=cf9ffbc895324c999d179eab327e1949, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.618351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d2396-844f-489d-9b9f-f9ca1eac978b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.632435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.633136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.633646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.633982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.639799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 943/1886] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:36 2022] PUT /identity/v3/projects/7b1e99faded24e9f83c11402ba92358f/users/cf9ffbc895324c999d179eab327e1949/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:36.641753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.642040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.642357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.647484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.649768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.659188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 944/1887] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:36.659850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.669120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0f437-8edb-4b61-b4a4-5870cb06b678 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.675183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] MFA Rules not processed for user `b60210cce9bf4b2b8119d31e8700e3dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:36.676525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.740298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.741547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.741975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.742328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.743302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.745717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.759428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 945/1888] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:36.763753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.780717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 944/1889] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:36.781238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab6a8-4325-4c6e-8870-1ec1394b293e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.789221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a27ef6-5087-4b0b-a17c-38ac92fd6adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.790338 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a27ef6-5087-4b0b-a17c-38ac92fd6adc admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.790904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a27ef6-5087-4b0b-a17c-38ac92fd6adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.792123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a27ef6-5087-4b0b-a17c-38ac92fd6adc admin admin] PATH_INFO: `/v3/projects/7b1e99faded24e9f83c11402ba92358f/users/cf9ffbc895324c999d179eab327e1949/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.792984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.794676 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a27ef6-5087-4b0b-a17c-38ac92fd6adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1e99faded24e9f83c11402ba92358f, user_id=cf9ffbc895324c999d179eab327e1949, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:36.794801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.797919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a27ef6-5087-4b0b-a17c-38ac92fd6adc admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:36.801004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:36.813949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 946/1890] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:36 2022] PUT /identity/v3/projects/7b1e99faded24e9f83c11402ba92358f/users/cf9ffbc895324c999d179eab327e1949/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:36.825882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.826441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.827276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.834401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:36.864077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] MFA Rules not processed for user `cf9ffbc895324c999d179eab327e1949`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:36.865393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.879624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db8cba-a341-42b1-b445-a940d70c0150 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.956004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.956725 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:36.957039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:36.957319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:36.960132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.960899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 947/1891] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:36.979305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:36.981039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:36.994820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34d60f-47f4-4f05-b39d-752c2dffd3c8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:36.998817 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:36.999651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:37.000211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:37.000581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:37.016919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:37.018391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:37.030601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 948/1892] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:37.040541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:37.042571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:37.056075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 945/1893] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:37.075257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:37.077117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:37.166859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bc107-2f98-4409-ac1a-3786f03237a2 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:37.177566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:37.178355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:37.178530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:37.178811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] PATH_INFO: `/v3/projects/7a7e5de723834100881245e603802e96` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:37.180151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7e5de723834100881245e603802e96)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:37.183005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:37.251859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7e5de723834100881245e603802e96', 'request_id': 'req-81130a83-0a99-441c-ac2f-155872aa264a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.278153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7e5de723834100881245e603802e96', 'request_id': 'req-81130a83-0a99-441c-ac2f-155872aa264a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.292200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7e5de723834100881245e603802e96', 'request_id': 'req-81130a83-0a99-441c-ac2f-155872aa264a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.294448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:37.309886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] The token cache is being invalidate because project 7a7e5de723834100881245e603802e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:37.310821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.311619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.313676 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 949/1894] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:37 2022] DELETE /identity/v3/projects/7a7e5de723834100881245e603802e96 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:37.317576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:37.330092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dde94-e012-455d-9b5b-96d0f953f644 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:37.406876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:37.407977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:37.408271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:37.408557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] PATH_INFO: `/v3/users/408163a245c3441fa4b432d4257ee5fb` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:37.410005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=408163a245c3441fa4b432d4257ee5fb)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:37.411997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:37.472356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:37.482751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '408163a245c3441fa4b432d4257ee5fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.483424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '408163a245c3441fa4b432d4257ee5fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.497465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '408163a245c3441fa4b432d4257ee5fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.498326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '408163a245c3441fa4b432d4257ee5fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.513503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '408163a245c3441fa4b432d4257ee5fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:37.530307 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 946/1895] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:37 2022] DELETE /identity/v3/users/408163a245c3441fa4b432d4257ee5fb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:37.825870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:37.827685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:37.898846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81130a83-0a99-441c-ac2f-155872aa264a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:37.929832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:37.931249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:37.975940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9ffbc895324c999d179eab327e1949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b1e99faded24e9f83c11402ba92358f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:37.976619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:37.976909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:37.977192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:37.978021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:37.978367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:37.980270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:37.992086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 950/1896] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:37.996001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:37.997586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.002846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf77f1-6615-4d43-9fd2-2f51be6d58b5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.007389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.008280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.008565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.008843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] PATH_INFO: `/v3/projects/52d9019a270f42879deb54e79fda3abc` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.009993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3ffad-02f2-4a85-9c93-e4bcd9c0e105 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.010538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d9019a270f42879deb54e79fda3abc)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.012495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.014730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9ffbc895324c999d179eab327e1949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b1e99faded24e9f83c11402ba92358f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.015404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.015690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.016028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.016789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.018673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.046115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 951/1897] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:37 2022] POST /identity/v3/projects => generated 353 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:38.050094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.060599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d9019a270f42879deb54e79fda3abc', 'request_id': 'req-e60ccf04-c054-48bd-8306-91ab38dea25a'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.066769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d9019a270f42879deb54e79fda3abc', 'request_id': 'req-e60ccf04-c054-48bd-8306-91ab38dea25a'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.073084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d9019a270f42879deb54e79fda3abc', 'request_id': 'req-e60ccf04-c054-48bd-8306-91ab38dea25a'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.087112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] The token cache is being invalidate because project 52d9019a270f42879deb54e79fda3abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:38.087406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.088615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e60ccf04-c054-48bd-8306-91ab38dea25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.090204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 947/1898] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:37 2022] DELETE /identity/v3/projects/52d9019a270f42879deb54e79fda3abc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:38.095000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d225d-69b8-4475-a0a1-2f16c6abfd31 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.095229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d225d-69b8-4475-a0a1-2f16c6abfd31 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.095507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d225d-69b8-4475-a0a1-2f16c6abfd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.114553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3a6d225d-69b8-4475-a0a1-2f16c6abfd31 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:38.115779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3a6d225d-69b8-4475-a0a1-2f16c6abfd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.121661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0fe1f-026f-485f-ba71-25402d396bd7 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.149966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:38.163019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:38.187551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 948/1899] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:38.196294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.197633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.197633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.201951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9ffbc895324c999d179eab327e1949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b1e99faded24e9f83c11402ba92358f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.202628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.202925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.203207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.203981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:38.204296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.206232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.218607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:38.222708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 952/1900] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:24:38.222842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.229942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 949/1901] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:38.234159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.245404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8a122-fa73-4e90-b6f3-fb839a322627 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.250101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.250757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.251130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.251307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.252071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:38.252382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.253981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.266491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 953/1902] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:38.267484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.268916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.270518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.279935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a674ad7-67fb-44be-9687-0c6f545cd404 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.282116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367142d-d593-4b4b-9cf0-0fce5e128aa3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.284611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.285250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.285630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.285906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.286589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.286928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9ffbc895324c999d179eab327e1949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b1e99faded24e9f83c11402ba92358f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.287778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.288097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.288377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] PATH_INFO: `/v3/projects/46af0318a0d543acb3bfc394edd8f26d` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.288516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.289557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorizing `identity:delete_project(project_id=46af0318a0d543acb3bfc394edd8f26d)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.291476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.316578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 954/1903] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:38.320813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.338854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46af0318a0d543acb3bfc394edd8f26d', 'request_id': 'req-bb711585-6ee0-483b-b501-48a4aaef8c31'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.345131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46af0318a0d543acb3bfc394edd8f26d', 'request_id': 'req-bb711585-6ee0-483b-b501-48a4aaef8c31'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.351108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46af0318a0d543acb3bfc394edd8f26d', 'request_id': 'req-bb711585-6ee0-483b-b501-48a4aaef8c31'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.363769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] The token cache is being invalidate because project 46af0318a0d543acb3bfc394edd8f26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:38.364127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.364811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.366422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 950/1904] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:38 2022] DELETE /identity/v3/projects/46af0318a0d543acb3bfc394edd8f26d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:38.385545 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cd4cd-538d-4aec-a935-021971b9e347 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.445232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.445900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.446185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.446458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.447690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.450098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.472034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 955/1905] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:38.476293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.477722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.490966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb711585-6ee0-483b-b501-48a4aaef8c31 tempest-VolumeQuotaClassesTest-1043383919 tempest-VolumeQuotaClassesTest-1043383919-project-admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.496161 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.496827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.497109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.497416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.498105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.499691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.510583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 951/1906] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:38.514625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.524466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9956d5e-df2b-4cf7-83d6-c2e7e3919cb6 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.529144 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.529803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.530087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.530368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.531100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.532740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.543915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 956/1907] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:38.547019 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.559752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1592178-4bad-4123-9ffc-06f7f0efbfaa admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.565664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927f4ec-93e3-49d1-b451-2c17396f91ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.567553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f4ec-93e3-49d1-b451-2c17396f91ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.567882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f4ec-93e3-49d1-b451-2c17396f91ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.568176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f4ec-93e3-49d1-b451-2c17396f91ce admin admin] PATH_INFO: `/v3/projects/210740092f7a43c0a9c86f26fa2f3b38/users/7e8b449fddda4a5297673b624f8c81dc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.575979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.577787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.589041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927f4ec-93e3-49d1-b451-2c17396f91ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=210740092f7a43c0a9c86f26fa2f3b38, user_id=7e8b449fddda4a5297673b624f8c81dc, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.590929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54154e9-5117-43ab-aa55-d9d604defe3e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.592279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927f4ec-93e3-49d1-b451-2c17396f91ce admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.595903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.596947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.597308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.597656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] PATH_INFO: `/v3/users/589746874d8a43fa84541d354033b720` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.599326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] RBAC: Authorizing `identity:delete_user(user_id=589746874d8a43fa84541d354033b720)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.601698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.603667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:38.610024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 952/1908] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:38 2022] PUT /identity/v3/projects/210740092f7a43c0a9c86f26fa2f3b38/users/7e8b449fddda4a5297673b624f8c81dc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:38.621228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.621515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.621799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.650872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] MFA Rules not processed for user `7e8b449fddda4a5297673b624f8c81dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:38.652124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.685145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589746874d8a43fa84541d354033b720'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.686815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589746874d8a43fa84541d354033b720'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.690984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589746874d8a43fa84541d354033b720'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.704594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589746874d8a43fa84541d354033b720'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.712596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3159e544-ac9b-4c8c-8b84-01971039c36e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589746874d8a43fa84541d354033b720'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:38.727569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 957/1909] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:38 2022] DELETE /identity/v3/users/589746874d8a43fa84541d354033b720 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:38.732377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.732952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.733515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.735395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 953/1910] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:38.739887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.741221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.757199 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:38.758526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.762723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:38.817136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d6f16-3e55-4cc4-b8b7-e2a2f0fdebaf None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.826406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 958/1911] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:38.836822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.838185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:38.878488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.879135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.879422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.879703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.880465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.882378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.899792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c571f3-2e21-4c35-a6eb-afa325e8a8e0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.904356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.905187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.906170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.906170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.910143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 954/1912] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/projects => generated 389 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:38.914317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:38.924596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.926658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:38.938081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 959/1913] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:38.983120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d185c3-4c0d-43a8-b370-b0cb948b4686 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:38.987834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:38.988519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:38.988809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:38.989089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:38.989935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:38.991904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.018202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 955/1914] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:38 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:39.020954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.032453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a0c92-0466-42ca-a2fa-f85fc2497c7b service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.037936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.038856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.039160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.039501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.040644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.042596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.052944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 960/1915] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:39.057246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.067672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86889fbf-ce6c-4510-b0b6-9312e29314c2 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.072301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.073181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.073456 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.073735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] PATH_INFO: `/v3/projects/f8d13afc38574561ba4ae2bf4836a546/users/254a12757410417f9fd62d87edfcf04f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.089858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d13afc38574561ba4ae2bf4836a546, user_id=254a12757410417f9fd62d87edfcf04f, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.092575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.107325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 956/1916] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:39 2022] PUT /identity/v3/projects/f8d13afc38574561ba4ae2bf4836a546/users/254a12757410417f9fd62d87edfcf04f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:39.111659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.122966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44906528-665b-445d-bfd4-bd3bfebdb276 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.130959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:39.183719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.184422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.184710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.184989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.185721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.187347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.197341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 961/1917] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:39.201843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.213332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eaa485-2a81-4c98-8354-67adbae88506 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.219212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083724f0-b0f4-4815-8b09-1c58ff1c105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.220255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083724f0-b0f4-4815-8b09-1c58ff1c105a admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.220545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083724f0-b0f4-4815-8b09-1c58ff1c105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.220826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083724f0-b0f4-4815-8b09-1c58ff1c105a admin admin] PATH_INFO: `/v3/projects/f8d13afc38574561ba4ae2bf4836a546/users/254a12757410417f9fd62d87edfcf04f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.223073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083724f0-b0f4-4815-8b09-1c58ff1c105a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d13afc38574561ba4ae2bf4836a546, user_id=254a12757410417f9fd62d87edfcf04f, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.226109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083724f0-b0f4-4815-8b09-1c58ff1c105a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.235404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.238236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:39.247676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 957/1918] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:39 2022] PUT /identity/v3/projects/f8d13afc38574561ba4ae2bf4836a546/users/254a12757410417f9fd62d87edfcf04f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:39.254163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62688d7-556c-4018-974a-8c614384f147 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.259215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.259549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.259858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.265849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:39.296919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] MFA Rules not processed for user `254a12757410417f9fd62d87edfcf04f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:39.298300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:39.331262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.332177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.332459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.332734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] PATH_INFO: `/v3/projects/50d2c30221cc419495a1a83f01858025` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.334015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d2c30221cc419495a1a83f01858025)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.336057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.375409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 958/1919] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:39.387303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.388679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:39.393662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d2c30221cc419495a1a83f01858025', 'request_id': 'req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.403148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d2c30221cc419495a1a83f01858025', 'request_id': 'req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.412492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d2c30221cc419495a1a83f01858025', 'request_id': 'req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.429215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] The token cache is being invalidate because project 50d2c30221cc419495a1a83f01858025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:39.429516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.430194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.432642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 962/1920] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:39 2022] DELETE /identity/v3/projects/50d2c30221cc419495a1a83f01858025 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:39.436609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.438123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:39.454859 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1cbeb-31e7-482d-a869-d92417ece049 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.482107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:39.505700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.506316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cf1ea-8aa9-45ac-9ce5-1b20e8685cb5 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.506432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.506625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.506911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.508848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:39.511579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.512481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.512761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.513033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] PATH_INFO: `/v3/users/cf9ffbc895324c999d179eab327e1949` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.514226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9ffbc895324c999d179eab327e1949)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.516180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.563469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9ffbc895324c999d179eab327e1949'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.564359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9ffbc895324c999d179eab327e1949'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.565116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9ffbc895324c999d179eab327e1949'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.573485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:39.577640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9ffbc895324c999d179eab327e1949'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.578336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.579801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.585015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8e27126d-af2b-47ec-92f4-4d35b77933e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9ffbc895324c999d179eab327e1949'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.589465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 959/1921] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:39.599813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 963/1922] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:39 2022] DELETE /identity/v3/users/cf9ffbc895324c999d179eab327e1949 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:39.721938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:39.723409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:39.737747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6449297-9f9a-48e3-8ea4-51bdb7a1955b service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:39.812474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:39.813812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.814106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.814391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] PATH_INFO: `/v3/users/7e8b449fddda4a5297673b624f8c81dc` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.815803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8b449fddda4a5297673b624f8c81dc)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:39.818295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:39.866765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8b449fddda4a5297673b624f8c81dc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.867411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8b449fddda4a5297673b624f8c81dc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.874890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8b449fddda4a5297673b624f8c81dc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.875750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8b449fddda4a5297673b624f8c81dc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.887853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8b449fddda4a5297673b624f8c81dc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:39.902887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 960/1923] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:39 2022] DELETE /identity/v3/users/7e8b449fddda4a5297673b624f8c81dc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:39.907925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:39.907925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:39.908247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:39.929973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:39.931794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.000515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.000515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.016020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 964/1924] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:40.024213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.025710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.072910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a5d7f-56c9-4920-a861-b5d9cf8991d0 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.077697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.078632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.079106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.079455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] PATH_INFO: `/v3/projects/7b1e99faded24e9f83c11402ba92358f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.080993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1e99faded24e9f83c11402ba92358f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.082965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.092803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93eb61-1a1f-4104-b86e-d8277cba59c6 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.097441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374d056-a521-4221-b143-05bc0ca22ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.098105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374d056-a521-4221-b143-05bc0ca22ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.098540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374d056-a521-4221-b143-05bc0ca22ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.098826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374d056-a521-4221-b143-05bc0ca22ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.100835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:40.109762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374d056-a521-4221-b143-05bc0ca22ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.111080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374d056-a521-4221-b143-05bc0ca22ca0 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.118648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:40.125540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 965/1925] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:40.145904 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1e99faded24e9f83c11402ba92358f', 'request_id': 'req-70d4e87c-dca4-495d-8a9a-5b2e419719bc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.152862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1e99faded24e9f83c11402ba92358f', 'request_id': 'req-70d4e87c-dca4-495d-8a9a-5b2e419719bc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.160531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1e99faded24e9f83c11402ba92358f', 'request_id': 'req-70d4e87c-dca4-495d-8a9a-5b2e419719bc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.178644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] The token cache is being invalidate because project 7b1e99faded24e9f83c11402ba92358f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:40.179277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.180204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-70d4e87c-dca4-495d-8a9a-5b2e419719bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.183002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 961/1926] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:39 2022] DELETE /identity/v3/projects/7b1e99faded24e9f83c11402ba92358f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:40.188300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e449aaf-46b3-4dc6-a08d-22d7bbd1ad60 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.188968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e449aaf-46b3-4dc6-a08d-22d7bbd1ad60 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.188968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e449aaf-46b3-4dc6-a08d-22d7bbd1ad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.211498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2e449aaf-46b3-4dc6-a08d-22d7bbd1ad60 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:40.212904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2e449aaf-46b3-4dc6-a08d-22d7bbd1ad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.279460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 966/1927] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:40.287270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56143b-d333-484d-8b2c-14c3f8b8bb99 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.287557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56143b-d333-484d-8b2c-14c3f8b8bb99 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.287864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56143b-d333-484d-8b2c-14c3f8b8bb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.307501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0a56143b-d333-484d-8b2c-14c3f8b8bb99 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:40.308761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0a56143b-d333-484d-8b2c-14c3f8b8bb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.317368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 962/1928] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:40.321460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e449aaf-46b3-4dc6-a08d-22d7bbd1ad60 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.332072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e449aaf-46b3-4dc6-a08d-22d7bbd1ad60 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.336696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.337357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.337683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.337963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.338698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:40.339004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.340646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.352149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 967/1929] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:40.356933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56143b-d333-484d-8b2c-14c3f8b8bb99 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.365841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56143b-d333-484d-8b2c-14c3f8b8bb99 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.371089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.371738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.372063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.372342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.373048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.374953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.397594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.399369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.405123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 963/1930] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/projects => generated 407 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:40.405984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.418557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b11d2-54d5-429b-a3fc-8a344bcbcd00 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.483265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.483471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335374a5-f8dd-402d-b114-2c5fbd75ca9a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.484300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.485129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.485129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] PATH_INFO: `/v3/projects/210740092f7a43c0a9c86f26fa2f3b38` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.486358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=210740092f7a43c0a9c86f26fa2f3b38)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.488803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.488959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.489167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.489536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.489884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.490653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.492671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.518507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 964/1931] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/users => generated 304 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:40.522774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.535712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7720b-f6a7-4da8-b83a-649fe1c3a71b admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.543934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210740092f7a43c0a9c86f26fa2f3b38', 'request_id': 'req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.553422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210740092f7a43c0a9c86f26fa2f3b38', 'request_id': 'req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.562644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210740092f7a43c0a9c86f26fa2f3b38', 'request_id': 'req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.579550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] The token cache is being invalidate because project 210740092f7a43c0a9c86f26fa2f3b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:40.579874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.580786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.582651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 968/1932] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:40 2022] DELETE /identity/v3/projects/210740092f7a43c0a9c86f26fa2f3b38 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:40.586720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.588860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.600583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.601267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.601560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.601846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.602594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.604807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.610554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:40.612553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:40.623535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 965/1933] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:40.624494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.627339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.643108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904b69a-2246-43b1-bf5b-7d223e0567f0 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.649717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.651280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.651613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.652090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.652892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.655941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.674192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03be5ea-4ac7-4e08-ab96-4a5a86eee81d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.676284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 966/1934] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:40.677595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.680368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.681294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.681586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.681871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] PATH_INFO: `/v3/users/254a12757410417f9fd62d87edfcf04f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.683216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] RBAC: Authorizing `identity:delete_user(user_id=254a12757410417f9fd62d87edfcf04f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.685409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.688164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cb059-edd8-4a95-a006-8051db4f3e0d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.695153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9f9be-7773-456d-8216-85d332a17ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.696049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f9be-7773-456d-8216-85d332a17ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.696336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f9be-7773-456d-8216-85d332a17ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.696613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f9be-7773-456d-8216-85d332a17ea4 admin admin] PATH_INFO: `/v3/projects/471ac1f6b3fd4c5680bd394dd3eb0426/users/661177ac6b434ac78db74e6ac386a161/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.727893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9f9be-7773-456d-8216-85d332a17ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=471ac1f6b3fd4c5680bd394dd3eb0426, user_id=661177ac6b434ac78db74e6ac386a161, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.731182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9f9be-7773-456d-8216-85d332a17ea4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.756975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 967/1935] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:40 2022] PUT /identity/v3/projects/471ac1f6b3fd4c5680bd394dd3eb0426/users/661177ac6b434ac78db74e6ac386a161/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:40.757572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254a12757410417f9fd62d87edfcf04f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.757572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254a12757410417f9fd62d87edfcf04f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.757572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254a12757410417f9fd62d87edfcf04f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.760873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182427b3-3c34-499b-ad77-a3aded3f63a9 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.761171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182427b3-3c34-499b-ad77-a3aded3f63a9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.761455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182427b3-3c34-499b-ad77-a3aded3f63a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.773856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254a12757410417f9fd62d87edfcf04f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.787320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254a12757410417f9fd62d87edfcf04f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.804762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-182427b3-3c34-499b-ad77-a3aded3f63a9 None None] MFA Rules not processed for user `661177ac6b434ac78db74e6ac386a161`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:40.806306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-182427b3-3c34-499b-ad77-a3aded3f63a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.810446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 969/1936] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:40 2022] DELETE /identity/v3/users/254a12757410417f9fd62d87edfcf04f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:40.914302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 968/1937] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:40.921295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.924252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.069782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94919e5e-b183-4a53-a791-243148f9ffda admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.077601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.080841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.082796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.084038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.086390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.089059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.133813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 970/1938] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/projects => generated 407 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:41.138189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182427b3-3c34-499b-ad77-a3aded3f63a9 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.194140 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.195883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.238863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:41.289261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182427b3-3c34-499b-ad77-a3aded3f63a9 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.297349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77796cbf-69fb-4f0b-b7d0-c1ee65bcd448 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.303047 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.304286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.304912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.305300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] PATH_INFO: `/v3/projects/f8d13afc38574561ba4ae2bf4836a546` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.305439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.306316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.306611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.306897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.307099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d13afc38574561ba4ae2bf4836a546)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.307749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.309398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.310473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.341459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 969/1939] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/users => generated 310 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:41.346735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.359408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09021b-aca1-472f-94e0-2fa10b83c1f6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.373931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.375552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.375552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.375785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.376694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.378667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.391915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d13afc38574561ba4ae2bf4836a546', 'request_id': 'req-2dd31a98-09d2-468b-946e-bbc4c3834858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:41.392719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 970/1940] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:41.396700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.407808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d13afc38574561ba4ae2bf4836a546', 'request_id': 'req-2dd31a98-09d2-468b-946e-bbc4c3834858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:41.408473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe00ec-0dad-4c6f-9c8d-189484f1ca8b admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.414972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d13afc38574561ba4ae2bf4836a546', 'request_id': 'req-2dd31a98-09d2-468b-946e-bbc4c3834858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:41.428818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] The token cache is being invalidate because project f8d13afc38574561ba4ae2bf4836a546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:41.429004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:41.429677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2dd31a98-09d2-468b-946e-bbc4c3834858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:41.431732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 971/1941] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:41 2022] DELETE /identity/v3/projects/f8d13afc38574561ba4ae2bf4836a546 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:41.436188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed3eb4-3312-4a58-afe3-0afa3f999040 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.436474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed3eb4-3312-4a58-afe3-0afa3f999040 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.436751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed3eb4-3312-4a58-afe3-0afa3f999040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.443865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:41.457035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-40ed3eb4-3312-4a58-afe3-0afa3f999040 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:41.458840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-40ed3eb4-3312-4a58-afe3-0afa3f999040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.475881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.476765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.477060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.477337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] PATH_INFO: `/v3/projects/2d68b79d84f440de91939e59061a1fb4/users/7a07b43b993b406e819ed6215772b013/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.491646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d68b79d84f440de91939e59061a1fb4, user_id=7a07b43b993b406e819ed6215772b013, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.494683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.507529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 971/1942] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:41 2022] PUT /identity/v3/projects/2d68b79d84f440de91939e59061a1fb4/users/7a07b43b993b406e819ed6215772b013/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:41.511334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.512925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.517949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 972/1943] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:41.525878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163c3a-b5cc-47a5-8c32-f0b956ef7f6e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.526290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163c3a-b5cc-47a5-8c32-f0b956ef7f6e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.526712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163c3a-b5cc-47a5-8c32-f0b956ef7f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.546130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-89163c3a-b5cc-47a5-8c32-f0b956ef7f6e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:41.547372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-89163c3a-b5cc-47a5-8c32-f0b956ef7f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.575420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35459f-067d-41e9-8302-d648bb34949e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.580650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.581324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.581610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.581891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.582702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.584416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.593699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:41.594660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 972/1944] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:41.598775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.608655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 973/1945] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:41.609724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977977e-7cd7-4922-b569-a114aea21cf9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.612490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89163c3a-b5cc-47a5-8c32-f0b956ef7f6e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.614415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8189a4-894e-445d-8b87-b4d8138adaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.615290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8189a4-894e-445d-8b87-b4d8138adaba admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.615574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8189a4-894e-445d-8b87-b4d8138adaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.615884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8189a4-894e-445d-8b87-b4d8138adaba admin admin] PATH_INFO: `/v3/projects/2d68b79d84f440de91939e59061a1fb4/users/7a07b43b993b406e819ed6215772b013/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.617963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8189a4-894e-445d-8b87-b4d8138adaba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d68b79d84f440de91939e59061a1fb4, user_id=7a07b43b993b406e819ed6215772b013, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.620650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8189a4-894e-445d-8b87-b4d8138adaba admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.621127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89163c3a-b5cc-47a5-8c32-f0b956ef7f6e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.625906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.626541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.626820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.627098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.627835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:41.628156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.629726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.632511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 973/1946] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:41 2022] PUT /identity/v3/projects/2d68b79d84f440de91939e59061a1fb4/users/7a07b43b993b406e819ed6215772b013/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:41.643768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.644083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.644363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.646995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 974/1947] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:41.650639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.662661 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf91e51-c081-4e59-9934-1d6b3046ae86 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.676722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] MFA Rules not processed for user `7a07b43b993b406e819ed6215772b013`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:41.677840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.725742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.726370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.726629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.726901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.727598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.730066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.758183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 975/1948] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:41.762523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.767980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 974/1949] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:41.776341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.779695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.813152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:41.847717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f1ad0-8b23-4ff2-8fe1-80bf8085a472 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.852903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3639af0-a1c8-4fff-9a77-a7172be4a885 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.858347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.859051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.859345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.859616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.860349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.862253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.882538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 976/1950] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/users => generated 284 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:41.886663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.895952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e775860-b21a-44f4-955f-1fe29e0748be admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.897362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a2108-d0a7-427b-b4c1-2add289b5ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.898011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2108-d0a7-427b-b4c1-2add289b5ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.898297 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2108-d0a7-427b-b4c1-2add289b5ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.898574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2108-d0a7-427b-b4c1-2add289b5ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.900707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.901365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.901643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.901922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.902649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.904288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.917564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 977/1951] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:41.919322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.929351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f5f5e-3aa6-4f10-8c8c-5825a421c21a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.934552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.935205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.935486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.935766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.936528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.938376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.948933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 978/1952] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:41.953057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.963059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18585430-b8e1-4c57-b0dd-1d4004a4be8b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.967141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a2108-d0a7-427b-b4c1-2add289b5ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.967906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.968647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a2108-d0a7-427b-b4c1-2add289b5ca3 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.968832 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.969072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.969347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] PATH_INFO: `/v3/projects/53dc8e079e6c44e3b63f3fca72c265cc/users/c8841fa7e3f74ef8872f813551a2de3b/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.980149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 975/1953] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:41.985031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=53dc8e079e6c44e3b63f3fca72c265cc, user_id=c8841fa7e3f74ef8872f813551a2de3b, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.987755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.001185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 979/1954] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:41 2022] PUT /identity/v3/projects/53dc8e079e6c44e3b63f3fca72c265cc/users/c8841fa7e3f74ef8872f813551a2de3b/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:42.012273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c737a-f730-47d0-a6c4-b91963cd95f6 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.012564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c737a-f730-47d0-a6c4-b91963cd95f6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.012846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c737a-f730-47d0-a6c4-b91963cd95f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.042298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-050c737a-f730-47d0-a6c4-b91963cd95f6 None None] MFA Rules not processed for user `c8841fa7e3f74ef8872f813551a2de3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:42.043557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-050c737a-f730-47d0-a6c4-b91963cd95f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:42.113645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 976/1955] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:42.118192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.128400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:42.147546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c737a-f730-47d0-a6c4-b91963cd95f6 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.169273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:42.184607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ef8b6-2d98-4ef6-b653-0ad9aa4b11ae admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.189300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.189963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.190256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.190534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.191231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.193282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.204921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c737a-f730-47d0-a6c4-b91963cd95f6 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.209938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.210597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.210867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.211180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.213658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:42.224583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 980/1956] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:42 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:42.225602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.289872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.292003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.301416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a5db-7c3f-460d-94bc-1c05b5f3916f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.308030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 977/1957] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:42 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:42.309946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.310667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.311057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.311346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.312347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.314807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.338026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 981/1958] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:42 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:42.341928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.354110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9284-040a-4629-8110-97eb698fafed service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.359141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.360024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.360436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.360789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.361785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.364009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.375463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 978/1959] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:42.379445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.390266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825d3b-1717-4c5c-bb9a-b7bc5b2f2e9f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.395024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.395975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.396272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.396576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] PATH_INFO: `/v3/projects/8a765aeffc1e4efa877baaead0948468/users/11e692a8e8f64083bddfc7883851c690/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.412587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a765aeffc1e4efa877baaead0948468, user_id=11e692a8e8f64083bddfc7883851c690, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.415542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.433173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 982/1960] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:42 2022] PUT /identity/v3/projects/8a765aeffc1e4efa877baaead0948468/users/11e692a8e8f64083bddfc7883851c690/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:42.437290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.448395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62598e-b30e-4de2-862e-f0968e5a679c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.524209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26aa91-cbf0-4897-9038-bd8e700d92f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.524909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26aa91-cbf0-4897-9038-bd8e700d92f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.525693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26aa91-cbf0-4897-9038-bd8e700d92f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.525982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26aa91-cbf0-4897-9038-bd8e700d92f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.526852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26aa91-cbf0-4897-9038-bd8e700d92f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.531272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26aa91-cbf0-4897-9038-bd8e700d92f0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.543966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 979/1961] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:42.545915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.562002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3c012-8ab6-4315-9839-3aa050b10600 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.567034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.568031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.568311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.568598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] PATH_INFO: `/v3/projects/8a765aeffc1e4efa877baaead0948468/users/11e692a8e8f64083bddfc7883851c690/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.572800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a765aeffc1e4efa877baaead0948468, user_id=11e692a8e8f64083bddfc7883851c690, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.575681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.599757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 983/1962] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:42 2022] PUT /identity/v3/projects/8a765aeffc1e4efa877baaead0948468/users/11e692a8e8f64083bddfc7883851c690/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:42.612055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c9fd-ed03-490e-9b9a-fd76ae8e176b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.613366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c9fd-ed03-490e-9b9a-fd76ae8e176b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.614419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c9fd-ed03-490e-9b9a-fd76ae8e176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.657573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-bee1c9fd-ed03-490e-9b9a-fd76ae8e176b None None] MFA Rules not processed for user `11e692a8e8f64083bddfc7883851c690`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:42.659192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bee1c9fd-ed03-490e-9b9a-fd76ae8e176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:42.707521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:42.730446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 980/1963] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:42 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:42.742193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.794576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:42.800516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cd8c9-2093-460e-a4ee-febb909d8d59 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.804610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.805260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.805543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.805818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.815893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.817594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.839925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 984/1964] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:42 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:42.931633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1c9fd-ed03-490e-9b9a-fd76ae8e176b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.942190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1c9fd-ed03-490e-9b9a-fd76ae8e176b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.947362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.948722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.949202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.949537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:43.017717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:43.020574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:43.031251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 981/1965] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:49.360773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:49.369609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edab47-c33b-4998-bf10-040655300d64 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:49.373470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e88353-23f7-4e46-9ff4-81cbc4930d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:49.374103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e88353-23f7-4e46-9ff4-81cbc4930d5e service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:49.374399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e88353-23f7-4e46-9ff4-81cbc4930d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:49.374669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e88353-23f7-4e46-9ff4-81cbc4930d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:49.376473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-36e88353-23f7-4e46-9ff4-81cbc4930d5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:49.444207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e88353-23f7-4e46-9ff4-81cbc4930d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:49.445577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e88353-23f7-4e46-9ff4-81cbc4930d5e service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:49.457204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 985/1966] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:50.255773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:50.339165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79264df-86f7-45a6-9487-467efdf0c3ac service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:50.346504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:50.347563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:50.347995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:50.348467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] PATH_INFO: `/v3/users/c8841fa7e3f74ef8872f813551a2de3b` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:50.349822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8841fa7e3f74ef8872f813551a2de3b)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:50.352592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:50.416070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8841fa7e3f74ef8872f813551a2de3b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:50.416719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8841fa7e3f74ef8872f813551a2de3b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:50.424923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8841fa7e3f74ef8872f813551a2de3b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:50.425571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8841fa7e3f74ef8872f813551a2de3b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:50.440835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8841fa7e3f74ef8872f813551a2de3b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:50.454169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 982/1967] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:50 2022] DELETE /identity/v3/users/c8841fa7e3f74ef8872f813551a2de3b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:50.458944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:50.459472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:50.459554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:50.476469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:50.477693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:50.541853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 986/1968] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:50.551799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:50.553218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:50.559241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:50.614349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56575139-ce6c-4eb7-a42e-a8893542f9c3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:50.618028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:50.619227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:50.619519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:50.619792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:50.629182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:50.630687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:50.642054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 983/1969] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:51.104604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:51.106011 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:51.129344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae2c23-6d01-4d08-b5b5-da9fe3821ab1 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:51.135844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:51.136736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:51.137022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:51.137297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] PATH_INFO: `/v3/projects/53dc8e079e6c44e3b63f3fca72c265cc` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:51.139568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=53dc8e079e6c44e3b63f3fca72c265cc)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:51.141448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:51.222773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53dc8e079e6c44e3b63f3fca72c265cc', 'request_id': 'req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.238457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53dc8e079e6c44e3b63f3fca72c265cc', 'request_id': 'req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.247789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53dc8e079e6c44e3b63f3fca72c265cc', 'request_id': 'req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.261813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] The token cache is being invalidate because project 53dc8e079e6c44e3b63f3fca72c265cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:51.262109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.262786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.264554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 987/1970] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:51 2022] DELETE /identity/v3/projects/53dc8e079e6c44e3b63f3fca72c265cc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:51.268745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:51.279305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb23ed-ae7a-402e-ae02-a35fd46e2cf2 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:51.364324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:51.365226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:51.365520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:51.365801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] PATH_INFO: `/v3/users/11e692a8e8f64083bddfc7883851c690` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:51.367039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e692a8e8f64083bddfc7883851c690)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:51.369040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:51.446020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e692a8e8f64083bddfc7883851c690'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.447263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e692a8e8f64083bddfc7883851c690'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.458388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e692a8e8f64083bddfc7883851c690'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.459097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e692a8e8f64083bddfc7883851c690'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.499599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-dcf341bf-d776-4556-b1a3-3bb5654ed7bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e692a8e8f64083bddfc7883851c690'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.517837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 984/1971] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:51 2022] DELETE /identity/v3/users/11e692a8e8f64083bddfc7883851c690 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:52.107933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.109741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.125325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ed98-1ffb-4b58-89e3-329bd6f63cea admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.132188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:52.215357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.216509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.217012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.217118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] PATH_INFO: `/v3/projects/8a765aeffc1e4efa877baaead0948468` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.218490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a765aeffc1e4efa877baaead0948468)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.220507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.275728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a765aeffc1e4efa877baaead0948468', 'request_id': 'req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.284873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a765aeffc1e4efa877baaead0948468', 'request_id': 'req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.291760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a765aeffc1e4efa877baaead0948468', 'request_id': 'req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.304905 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] The token cache is being invalidate because project 8a765aeffc1e4efa877baaead0948468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:52.305271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.306040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ad774c4b-0c27-4fc1-9afa-e1b80c99876a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.307664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 988/1972] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:52 2022] DELETE /identity/v3/projects/8a765aeffc1e4efa877baaead0948468 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:52.312741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9721c-6e9e-46ea-8f4a-7a8843901e48 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.313222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9721c-6e9e-46ea-8f4a-7a8843901e48 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.313407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9721c-6e9e-46ea-8f4a-7a8843901e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.332263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-25d9721c-6e9e-46ea-8f4a-7a8843901e48 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:52.333519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-25d9721c-6e9e-46ea-8f4a-7a8843901e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.355559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:52.401091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 985/1973] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:52.408898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfee4bc-22c9-47a6-9487-2fa19a2d4687 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.409439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfee4bc-22c9-47a6-9487-2fa19a2d4687 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.409797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfee4bc-22c9-47a6-9487-2fa19a2d4687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.429766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0cfee4bc-22c9-47a6-9487-2fa19a2d4687 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:52.431295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0cfee4bc-22c9-47a6-9487-2fa19a2d4687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.442190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 989/1974] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:52.443612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9721c-6e9e-46ea-8f4a-7a8843901e48 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.455428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9721c-6e9e-46ea-8f4a-7a8843901e48 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.460973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.461768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.462718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.463152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.464113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:52.464690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.466479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.479511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 986/1975] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:52.484049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfee4bc-22c9-47a6-9487-2fa19a2d4687 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.495089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfee4bc-22c9-47a6-9487-2fa19a2d4687 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.500728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.501428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.501747 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.502062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.502840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.504789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.535839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 990/1976] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:52.540570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.558663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76935ace-8b9c-42f7-b588-5dba6ca150c7 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.575078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.577800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.649257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.650584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.650584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.650897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.651690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.654051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.690323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 987/1977] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:52.695247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.718359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219100e3-c1f2-437e-9bdc-c5cc5f6440f4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.739769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.739769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.739769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.739769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.740142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.742116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.760190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 988/1978] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:52.764365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.768851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d659d-f1ce-4c6f-9695-97b82c2c27f3 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.777722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.777722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.777981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.778075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] PATH_INFO: `/v3/users/661177ac6b434ac78db74e6ac386a161` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.779162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deac04c-f596-4ba3-b60b-4ca93a9ff51f admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.779527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661177ac6b434ac78db74e6ac386a161)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.784919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.786379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.787089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.787520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.787984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.788898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.791611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.808328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:52.830796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 989/1979] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:52.834072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.849427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af033893-aab4-478e-a3cc-7ce524da6081 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.865499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a45228-eda0-432f-9000-21d20da1471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.866422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a45228-eda0-432f-9000-21d20da1471b admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.867164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a45228-eda0-432f-9000-21d20da1471b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.867447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a45228-eda0-432f-9000-21d20da1471b admin admin] PATH_INFO: `/v3/projects/569038bf676a460c9fa21558c61132c2/users/c22c34bc59fc4f6cb9d4449beb250a77/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.876301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661177ac6b434ac78db74e6ac386a161'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.881121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661177ac6b434ac78db74e6ac386a161'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.881806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661177ac6b434ac78db74e6ac386a161'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.901764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a45228-eda0-432f-9000-21d20da1471b admin admin] RBAC: Authorizing `identity:create_grant(project_id=569038bf676a460c9fa21558c61132c2, user_id=c22c34bc59fc4f6cb9d4449beb250a77, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.905217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661177ac6b434ac78db74e6ac386a161'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.907323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a45228-eda0-432f-9000-21d20da1471b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.922552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-78dd1d75-4b80-4644-acd0-d92fd9d31652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661177ac6b434ac78db74e6ac386a161'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.938176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 990/1980] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:52 2022] PUT /identity/v3/projects/569038bf676a460c9fa21558c61132c2/users/c22c34bc59fc4f6cb9d4449beb250a77/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:52.972916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.973210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.975382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.980572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 991/1981] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:52 2022] DELETE /identity/v3/users/661177ac6b434ac78db74e6ac386a161 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:52.986123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.986367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.986648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:53.018527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:53.019912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:53.031830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:53.033150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:53.106046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:53.130436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 991/1982] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:53.141975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:53.143393 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:53.144539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 992/1983] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:53.159323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:53.161194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:53.249093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b2498-54db-4478-b178-5104388d67f4 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:53.254191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:53.255014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:53.257326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:53.257648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:53.263210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2c82c-b391-463a-b346-8f6d424b9ac7 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:53.267436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:53.268277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:53.268698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:53.268989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:53.270215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:53.272138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:53.280573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:53.282213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:53.283576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 992/1984] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:53.298238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 993/1985] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:53.715442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:53.717888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:53.737503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2c9fb-c563-4c6c-af38-9a28343f9a3f service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:53.743801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:53.744728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:53.745015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:53.745548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] PATH_INFO: `/v3/projects/471ac1f6b3fd4c5680bd394dd3eb0426` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:53.747524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] RBAC: Authorizing `identity:delete_project(project_id=471ac1f6b3fd4c5680bd394dd3eb0426)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:53.750427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:53.819962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471ac1f6b3fd4c5680bd394dd3eb0426', 'request_id': 'req-afada256-9a3c-4203-941d-cded0b0473da'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:53.828545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471ac1f6b3fd4c5680bd394dd3eb0426', 'request_id': 'req-afada256-9a3c-4203-941d-cded0b0473da'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:53.835001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471ac1f6b3fd4c5680bd394dd3eb0426', 'request_id': 'req-afada256-9a3c-4203-941d-cded0b0473da'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:53.848008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] The token cache is being invalidate because project 471ac1f6b3fd4c5680bd394dd3eb0426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:53.848363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:53.848919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:53.851886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 993/1986] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:53 2022] DELETE /identity/v3/projects/471ac1f6b3fd4c5680bd394dd3eb0426 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:53.855333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:53.871691 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bfd41-d6fd-4444-9b19-fb6b1a7d7390 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:53.948171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:53.999759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:54.000903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:54.001355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:54.001790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] PATH_INFO: `/v3/users/7a07b43b993b406e819ed6215772b013` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:54.003636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a07b43b993b406e819ed6215772b013)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:54.005819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:54.112425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a07b43b993b406e819ed6215772b013'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.113126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a07b43b993b406e819ed6215772b013'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.114324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a07b43b993b406e819ed6215772b013'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.145388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a07b43b993b406e819ed6215772b013'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.156737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b294cfe-41e7-4371-ab12-4d81b8fbe298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a07b43b993b406e819ed6215772b013'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.184642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 994/1987] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:53 2022] DELETE /identity/v3/users/7a07b43b993b406e819ed6215772b013 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:54.491154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:54.493630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:54.530142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afada256-9a3c-4203-941d-cded0b0473da admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:54.609891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:54.610275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:54.610640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:54.673990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] MFA Rules not processed for user `c22c34bc59fc4f6cb9d4449beb250a77`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:54.676803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:54.681561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:54.683298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:54.684988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:54.685294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] PATH_INFO: `/v3/users/ebef406a97ad48a18202b4f769e9d39c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:54.692526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebef406a97ad48a18202b4f769e9d39c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:54.696954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:54.750871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:54.823908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebef406a97ad48a18202b4f769e9d39c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.828863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebef406a97ad48a18202b4f769e9d39c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.846241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebef406a97ad48a18202b4f769e9d39c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.846968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebef406a97ad48a18202b4f769e9d39c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.866812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-94dcd928-e724-4803-835c-0c5567e1c791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebef406a97ad48a18202b4f769e9d39c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:54.872183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 995/1988] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:54.879573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:54.882179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:54.890612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 994/1989] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:54 2022] DELETE /identity/v3/users/ebef406a97ad48a18202b4f769e9d39c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:54.898218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3e073-1de3-4278-94a0-218b97da772e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:54.898768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3e073-1de3-4278-94a0-218b97da772e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:54.900239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3e073-1de3-4278-94a0-218b97da772e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:54.911468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b384f4f-e3de-4e89-90b8-9cc34e783c1c None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:54.968029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-58f3e073-1de3-4278-94a0-218b97da772e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:54.969337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-58f3e073-1de3-4278-94a0-218b97da772e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:55.001597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:55.038777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.039675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.040495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.041334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.042769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.045488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.069233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 995/1990] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:55.086080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 996/1991] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:54 2022] POST /identity/v3/projects => generated 373 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:55.091076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3e073-1de3-4278-94a0-218b97da772e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:55.106677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3e073-1de3-4278-94a0-218b97da772e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:55.111474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:55.114232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:55.176270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.177198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.177585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.177965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.178903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.181330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.192105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea627a-4c92-4861-a2f3-9dfefc58049a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:55.197342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.198276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.199932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.201783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.208932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 996/1992] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:55 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:55.213239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:55.219630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.221710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.224715 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7555d-9af0-460d-a194-0571e2fc9173 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:55.230665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.231871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.232241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.232773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.233748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.236418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.237431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 997/1993] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:55.249011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 997/1994] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:55.253325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:55.264367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380582-59d3-4330-ac6f-460d65738d0f service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:55.271016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.272413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.272808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.273753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] PATH_INFO: `/v3/projects/a810ff297df4432a9e60abf985ce063a/users/1b1c67071d86461c8c3820fdd2701316/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.298293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a810ff297df4432a9e60abf985ce063a, user_id=1b1c67071d86461c8c3820fdd2701316, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.302300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.325650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 998/1995] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:55 2022] PUT /identity/v3/projects/a810ff297df4432a9e60abf985ce063a/users/1b1c67071d86461c8c3820fdd2701316/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:55.330168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:55.341943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad2ff5-47a3-496a-9f0f-970caa5786d1 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:55.370517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:55.425512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.426519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.426867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.427211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.428070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.429736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.444449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 998/1996] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:55.447665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:55.463114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eea461-0e9f-401e-b089-227a3c5f1ca3 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:55.468093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9e6e4-fcdf-4e31-aab1-c103fc3ab452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.471040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9e6e4-fcdf-4e31-aab1-c103fc3ab452 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.471560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9e6e4-fcdf-4e31-aab1-c103fc3ab452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.472344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9e6e4-fcdf-4e31-aab1-c103fc3ab452 admin admin] PATH_INFO: `/v3/projects/a810ff297df4432a9e60abf985ce063a/users/1b1c67071d86461c8c3820fdd2701316/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.475632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9e6e4-fcdf-4e31-aab1-c103fc3ab452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a810ff297df4432a9e60abf985ce063a, user_id=1b1c67071d86461c8c3820fdd2701316, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.479875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9e6e4-fcdf-4e31-aab1-c103fc3ab452 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.502477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 999/1997] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:55 2022] PUT /identity/v3/projects/a810ff297df4432a9e60abf985ce063a/users/1b1c67071d86461c8c3820fdd2701316/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:55.606554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:55.609310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:55.637095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe8987-824f-4d4b-aa0f-83fd46e00ad7 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:55.751773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:55.752751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.753047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.753329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] PATH_INFO: `/v3/projects/2d68b79d84f440de91939e59061a1fb4` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.754553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d68b79d84f440de91939e59061a1fb4)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:55.756591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:55.824297 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d68b79d84f440de91939e59061a1fb4', 'request_id': 'req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:55.831898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d68b79d84f440de91939e59061a1fb4', 'request_id': 'req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:55.838683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d68b79d84f440de91939e59061a1fb4', 'request_id': 'req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:55.862243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] The token cache is being invalidate because project 2d68b79d84f440de91939e59061a1fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:55.862642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:55.863439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-db8c91ed-7e0a-45f6-9611-113bd9bbf5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:55.869035 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 999/1998] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:55 2022] DELETE /identity/v3/projects/2d68b79d84f440de91939e59061a1fb4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:55.871070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ebe21-0814-4a6b-bbe7-f9ad8ce6ccea None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:55.871351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ebe21-0814-4a6b-bbe7-f9ad8ce6ccea None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:55.871621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ebe21-0814-4a6b-bbe7-f9ad8ce6ccea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:55.892720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-406ebe21-0814-4a6b-bbe7-f9ad8ce6ccea None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:55.894111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-406ebe21-0814-4a6b-bbe7-f9ad8ce6ccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.013772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1000/1999] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:56.022448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64bb75-eb67-4184-a898-90807a7f3c56 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.022838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64bb75-eb67-4184-a898-90807a7f3c56 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.023217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64bb75-eb67-4184-a898-90807a7f3c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.047328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ff64bb75-eb67-4184-a898-90807a7f3c56 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:56.047328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ff64bb75-eb67-4184-a898-90807a7f3c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.059862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1000/2000] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:56.061851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ebe21-0814-4a6b-bbe7-f9ad8ce6ccea None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.072846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ebe21-0814-4a6b-bbe7-f9ad8ce6ccea None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.078785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.079592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.080159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.080509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.081249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:56.081666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.085999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.111783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1001/2001] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:56.115998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64bb75-eb67-4184-a898-90807a7f3c56 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.132010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64bb75-eb67-4184-a898-90807a7f3c56 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.139290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.141852 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.158985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.159929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.160610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.161181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.162203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.164285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.167498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0ecc7-8695-4b81-acf5-b777160f162d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.174673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.175860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.176356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.176777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] PATH_INFO: `/v3/projects/6b9bfc3dc4ba47aeb34c60d5b5baf9d9` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.180124 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b9bfc3dc4ba47aeb34c60d5b5baf9d9)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.182534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.200113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:56.201747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1001/2002] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:56 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:56.208116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.255570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:56.260843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b9bfc3dc4ba47aeb34c60d5b5baf9d9', 'request_id': 'req-3c8e4493-c813-41bf-8c9e-aeae18355683'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.269781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b9bfc3dc4ba47aeb34c60d5b5baf9d9', 'request_id': 'req-3c8e4493-c813-41bf-8c9e-aeae18355683'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.277732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b9bfc3dc4ba47aeb34c60d5b5baf9d9', 'request_id': 'req-3c8e4493-c813-41bf-8c9e-aeae18355683'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.291766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] The token cache is being invalidate because project 6b9bfc3dc4ba47aeb34c60d5b5baf9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:56.292083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.292784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.294365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1002/2003] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:56 2022] DELETE /identity/v3/projects/6b9bfc3dc4ba47aeb34c60d5b5baf9d9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:56.299197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.300658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.322423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3284c8b-5b7e-4b0c-aedf-699cb72d6f90 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.379531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e4493-c813-41bf-8c9e-aeae18355683 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.385601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.386878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.387266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.387613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] PATH_INFO: `/v3/users/b60210cce9bf4b2b8119d31e8700e3dc` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.388895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60210cce9bf4b2b8119d31e8700e3dc)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.390873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.398725 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.400310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.400848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.402292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.403290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.405728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.433384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1002/2004] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:56 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:56.434782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.436799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.447297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60210cce9bf4b2b8119d31e8700e3dc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.448044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60210cce9bf4b2b8119d31e8700e3dc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.448683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60210cce9bf4b2b8119d31e8700e3dc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.449565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c059d5e-6ccc-4827-a336-f3d7dd309d00 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.455775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.456436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.456780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.456995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.457746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.459418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.463218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60210cce9bf4b2b8119d31e8700e3dc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.471873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1003/2005] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:56.472092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60210cce9bf4b2b8119d31e8700e3dc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:56.475299 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.477420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.490959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77f68b-2d54-4dd3-8dec-e60da7d2048e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.496787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1003/2006] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:56 2022] DELETE /identity/v3/users/b60210cce9bf4b2b8119d31e8700e3dc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:56.496987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fb322-3816-488b-9915-734959f4f725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.497839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fb322-3816-488b-9915-734959f4f725 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.497839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fb322-3816-488b-9915-734959f4f725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.498145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fb322-3816-488b-9915-734959f4f725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.498852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fb322-3816-488b-9915-734959f4f725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.500535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fb322-3816-488b-9915-734959f4f725 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.512429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1004/2007] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:56.517242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.542234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:56.598020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c673daa-b8a1-4c7b-b5c9-3087e90c0e8a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.603284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.604094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.604380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.604656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] PATH_INFO: `/v3/projects/7c04bae858fd49d1b060d30e8000f1b7/users/77a5a4342bab40108cd89fc1fde3a1cb/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.623576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c04bae858fd49d1b060d30e8000f1b7, user_id=77a5a4342bab40108cd89fc1fde3a1cb, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.626388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.642656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1004/2008] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:56 2022] PUT /identity/v3/projects/7c04bae858fd49d1b060d30e8000f1b7/users/77a5a4342bab40108cd89fc1fde3a1cb/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:56.659182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064581-7942-4f04-9f42-f3f7b4aa4e03 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.660406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064581-7942-4f04-9f42-f3f7b4aa4e03 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.661104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064581-7942-4f04-9f42-f3f7b4aa4e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.709357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-70064581-7942-4f04-9f42-f3f7b4aa4e03 None None] MFA Rules not processed for user `77a5a4342bab40108cd89fc1fde3a1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:56.711105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-70064581-7942-4f04-9f42-f3f7b4aa4e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.816667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.819587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:56.820947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1005/2009] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:56.825448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70064581-7942-4f04-9f42-f3f7b4aa4e03 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:56.872164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:56.956095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70064581-7942-4f04-9f42-f3f7b4aa4e03 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.963967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.964576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.964857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.965136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.965850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.968053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:56.976302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c040928-d4ac-4fd4-9b20-f8676c31d764 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:56.982760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:56.984397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:56.984885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:56.985497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] PATH_INFO: `/v3/projects/1d96d8b4813141d0bf8561570d6c291b` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:56.986984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d96d8b4813141d0bf8561570d6c291b)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:56.990129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.013385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1006/2010] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:56 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:57.014541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.092450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d96d8b4813141d0bf8561570d6c291b', 'request_id': 'req-c8110056-0358-4902-a2b7-d04b24ccfe3a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:57.107684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d96d8b4813141d0bf8561570d6c291b', 'request_id': 'req-c8110056-0358-4902-a2b7-d04b24ccfe3a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:57.117267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d96d8b4813141d0bf8561570d6c291b', 'request_id': 'req-c8110056-0358-4902-a2b7-d04b24ccfe3a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:57.134951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] The token cache is being invalidate because project 1d96d8b4813141d0bf8561570d6c291b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:57.135601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:57.137381 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8110056-0358-4902-a2b7-d04b24ccfe3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:57.139865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1005/2011] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:56 2022] DELETE /identity/v3/projects/1d96d8b4813141d0bf8561570d6c291b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:57.144198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5e44b-4ab6-4d37-a640-bc8bf95d296d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.144198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5e44b-4ab6-4d37-a640-bc8bf95d296d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.144198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5e44b-4ab6-4d37-a640-bc8bf95d296d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.216018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf6943-619c-40b0-b825-c5003dce1963 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.228297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-dde5e44b-4ab6-4d37-a640-bc8bf95d296d None None] MFA Rules not processed for user `1b1c67071d86461c8c3820fdd2701316`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:57.233472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dde5e44b-4ab6-4d37-a640-bc8bf95d296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:57.278150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:57.353006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1006/2012] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:57.353719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa831540-6c0e-4908-b4b5-e3ba8a835347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.354639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa831540-6c0e-4908-b4b5-e3ba8a835347 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.355002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.355289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.355530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa831540-6c0e-4908-b4b5-e3ba8a835347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.355637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.356066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa831540-6c0e-4908-b4b5-e3ba8a835347 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.356923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa831540-6c0e-4908-b4b5-e3ba8a835347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.360017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa831540-6c0e-4908-b4b5-e3ba8a835347 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.379624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:57.381464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:57.395673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1007/2013] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:57.396645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.401804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:57.409003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1007/2014] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/users => generated 289 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:57.411411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.412052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.413458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.437278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:57.438869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:57.452897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1008/2015] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:57.452897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.454487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:57.467004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd9b54-43aa-4791-85b1-abc937101663 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.472185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208eb12-b242-412e-ab31-21f590a9f8dd None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.473844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.474578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.474929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.475277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.476120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.477652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.478300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.478705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.480138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.480363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.483034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:57.489259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1009/2016] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:57.490075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.500383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4ef46-4ce6-4ee6-9dd5-fe750e65bc57 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.504938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.505590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.505878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.506155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.506880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:57.507189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.508979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.527323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1010/2017] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:57.527323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.535624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e04e-ce68-4dde-9953-00e97117352d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.543609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.544756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.545023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.545298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.546043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.548620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.560046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1011/2018] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:57.561073 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.563486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.565053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.566199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:57.571812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937639e-113c-40ba-863d-06c0ec3b7d76 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.575938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1008/2019] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:57.578636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.579124 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.580150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.580719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.581103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.582085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.584383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.602498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa231abb-5c1f-4fe0-b506-42b5ab686471 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.608513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c896df5-1817-45d3-acf9-f19c3cba09b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.609697 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c896df5-1817-45d3-acf9-f19c3cba09b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.610006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c896df5-1817-45d3-acf9-f19c3cba09b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.610492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c896df5-1817-45d3-acf9-f19c3cba09b1 admin admin] PATH_INFO: `/v3/projects/34b29b2cb9ab496fb9d179e22d8c8c05/users/200e6cb0657c449b93d08c4f9f4e8114/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.617094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1012/2020] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:57.621796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.635203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c896df5-1817-45d3-acf9-f19c3cba09b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b29b2cb9ab496fb9d179e22d8c8c05, user_id=200e6cb0657c449b93d08c4f9f4e8114, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.638085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c896df5-1817-45d3-acf9-f19c3cba09b1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.659071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1009/2021] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:57 2022] PUT /identity/v3/projects/34b29b2cb9ab496fb9d179e22d8c8c05/users/200e6cb0657c449b93d08c4f9f4e8114/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:57.669730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222b1a0-050a-4c5a-ba43-4f6e75d5729d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.670074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222b1a0-050a-4c5a-ba43-4f6e75d5729d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.670305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222b1a0-050a-4c5a-ba43-4f6e75d5729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.704340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-5222b1a0-050a-4c5a-ba43-4f6e75d5729d None None] MFA Rules not processed for user `200e6cb0657c449b93d08c4f9f4e8114`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:57.706467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5222b1a0-050a-4c5a-ba43-4f6e75d5729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:57.726799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5e0e5-26c2-4ef1-bd56-e2d96a63e4de admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.750486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:57.796098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1010/2022] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:57.800962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222b1a0-050a-4c5a-ba43-4f6e75d5729d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.817631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.818347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.818635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.818920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.819668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.821742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.847890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1013/2023] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:57.850867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.863082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36689eb5-fb9b-4fae-b5ea-29048ac3bf60 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.869950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.870483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.870799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.871143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.872180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.873860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.891257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222b1a0-050a-4c5a-ba43-4f6e75d5729d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.903127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1014/2024] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:57.907646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:57.918914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.919609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.920343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.920343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.923574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.925681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.938324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b696be-5974-4faf-8790-dc45d7b8e0f3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:57.947638 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:57.948371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:57.951400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:57.951732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:57.953425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:57.955457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:57.978086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1015/2025] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:57.982021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.001144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6cd7c-499b-48ba-a3f2-1a944a973702 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.002564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1011/2026] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:57 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:58.003473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.066731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:58.075789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93bd86-7a10-4c07-a99e-903753009c25 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.076105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f799f9-8cf4-40e3-9fbd-615f00a8e0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.077144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f799f9-8cf4-40e3-9fbd-615f00a8e0da admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.077451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f799f9-8cf4-40e3-9fbd-615f00a8e0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.077780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f799f9-8cf4-40e3-9fbd-615f00a8e0da admin admin] PATH_INFO: `/v3/projects/5297a69448d540f4ac0abfa001208ea6/users/1004f5ef090f44dabba4cfa0f18e73b5/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.083760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.084497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.084782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.085060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.085766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.088811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.094584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f799f9-8cf4-40e3-9fbd-615f00a8e0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5297a69448d540f4ac0abfa001208ea6, user_id=1004f5ef090f44dabba4cfa0f18e73b5, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.097306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f799f9-8cf4-40e3-9fbd-615f00a8e0da admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.111581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1012/2027] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:58 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:58.115172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1016/2028] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:57 2022] PUT /identity/v3/projects/5297a69448d540f4ac0abfa001208ea6/users/1004f5ef090f44dabba4cfa0f18e73b5/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:58.115403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.126882 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.127228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1a47c-24c4-4c59-bb3e-275c06c9b4a7 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.127326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.127410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.155425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] MFA Rules not processed for user `1004f5ef090f44dabba4cfa0f18e73b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:58.156663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:58.190304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:58.205794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.206473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.206752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.207028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.208099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.209926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.221940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1013/2029] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:58.226020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.237760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1017/2030] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:58.244770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe3da0-0db4-4cc3-9044-ade5c92b6580 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.250136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.251525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:58.252549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.253715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.254102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.254459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] PATH_INFO: `/v3/projects/2faafaaa37434aafb256f353fa512c10/users/fd73dfb4003d4247866793470be6674f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.273641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faafaaa37434aafb256f353fa512c10, user_id=fd73dfb4003d4247866793470be6674f, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.276977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.307205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1014/2031] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:58 2022] PUT /identity/v3/projects/2faafaaa37434aafb256f353fa512c10/users/fd73dfb4003d4247866793470be6674f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:58.311333 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.350718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50455468-6a37-4d83-a3bf-829f96956872 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.388455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd747ee6-0785-4a4c-b0ce-4867cd3871e4 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.394907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.395615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.395938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.396296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.397041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.398999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.412597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1015/2032] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:58.415040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.433240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e99e83-1a16-4e13-abac-16bd74f890f4 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.440658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676259f1-4717-44c6-9e06-8bb7e6055076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.441749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676259f1-4717-44c6-9e06-8bb7e6055076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.442240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676259f1-4717-44c6-9e06-8bb7e6055076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.442704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676259f1-4717-44c6-9e06-8bb7e6055076 admin admin] PATH_INFO: `/v3/projects/2faafaaa37434aafb256f353fa512c10/users/fd73dfb4003d4247866793470be6674f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.445259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676259f1-4717-44c6-9e06-8bb7e6055076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faafaaa37434aafb256f353fa512c10, user_id=fd73dfb4003d4247866793470be6674f, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.455166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676259f1-4717-44c6-9e06-8bb7e6055076 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.459173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.459853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.460146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.460425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.471213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1016/2033] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:58 2022] PUT /identity/v3/projects/2faafaaa37434aafb256f353fa512c10/users/fd73dfb4003d4247866793470be6674f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:58.482108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309a151-0976-417f-a26a-494d373b23b3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.482340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309a151-0976-417f-a26a-494d373b23b3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.482729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309a151-0976-417f-a26a-494d373b23b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.509926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:58.514993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f309a151-0976-417f-a26a-494d373b23b3 None None] MFA Rules not processed for user `fd73dfb4003d4247866793470be6674f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:58.516352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f309a151-0976-417f-a26a-494d373b23b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:58.533673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.535183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.545174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1018/2034] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:58.568237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.579455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb845df-d352-426e-b1dc-30c0dfd2339b service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.592481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1017/2035] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:58.604479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309a151-0976-417f-a26a-494d373b23b3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.633612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:58.637413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.638309 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.638601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.638905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] PATH_INFO: `/v3/users/1004f5ef090f44dabba4cfa0f18e73b5` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.640333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1004f5ef090f44dabba4cfa0f18e73b5)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.642314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.660850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309a151-0976-417f-a26a-494d373b23b3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.665155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.665810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.666087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.666374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.676397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.677824 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.687927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1018/2036] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:58.693463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1004f5ef090f44dabba4cfa0f18e73b5'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:58.694042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1004f5ef090f44dabba4cfa0f18e73b5'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:58.701700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1004f5ef090f44dabba4cfa0f18e73b5'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:58.702333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1004f5ef090f44dabba4cfa0f18e73b5'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:58.715906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e3df70f2-e0d0-4ae2-aec9-0233abf21d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1004f5ef090f44dabba4cfa0f18e73b5'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:58.729723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1019/2037] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:24:58 2022] DELETE /identity/v3/users/1004f5ef090f44dabba4cfa0f18e73b5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:58.731498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.733919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.734153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.734437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.740323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6db6-458b-41f0-8ab6-e8a1891eda0a service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.750932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:58.752169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:58.790980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.791759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.792187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.792438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.794631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:58.806568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:58.821201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1020/2038] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:58.828714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:58.831153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:58.859646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.861130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.870359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1019/2039] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:58.891683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9c31a-c32d-4413-b0bb-6095e3b0f78c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:58.895534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4a81-d90e-443f-b257-4e1a55f7a4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:58.896234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4a81-d90e-443f-b257-4e1a55f7a4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:58.896502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4a81-d90e-443f-b257-4e1a55f7a4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:58.896777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4a81-d90e-443f-b257-4e1a55f7a4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:58.906694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4a81-d90e-443f-b257-4e1a55f7a4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:58.908153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4a81-d90e-443f-b257-4e1a55f7a4f8 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:58.920893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1021/2040] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:59.210268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.213822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:59.241020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91b06-49bf-4b32-973f-f56c14455e60 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:59.256458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:59.257355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.257639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.258579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] PATH_INFO: `/v3/projects/5297a69448d540f4ac0abfa001208ea6` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.259653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5297a69448d540f4ac0abfa001208ea6)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:59.261647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:59.318403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5297a69448d540f4ac0abfa001208ea6', 'request_id': 'req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:59.328732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5297a69448d540f4ac0abfa001208ea6', 'request_id': 'req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:59.335480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5297a69448d540f4ac0abfa001208ea6', 'request_id': 'req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:59.347012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] The token cache is being invalidate because project 5297a69448d540f4ac0abfa001208ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:59.347302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:59.348000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f6f38d6d-f65d-463e-8c1f-e40ae4d510c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:59.349591 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1020/2041] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:24:59 2022] DELETE /identity/v3/projects/5297a69448d540f4ac0abfa001208ea6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:59.354544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cad64-54b6-4ad6-9a03-cfe9a4c16088 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.354855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cad64-54b6-4ad6-9a03-cfe9a4c16088 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.355134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cad64-54b6-4ad6-9a03-cfe9a4c16088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.373924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-763cad64-54b6-4ad6-9a03-cfe9a4c16088 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:59.375171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-763cad64-54b6-4ad6-9a03-cfe9a4c16088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:59.455480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1022/2042] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:59.457642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dbaef-b17b-4cb0-8ca8-e8edbce6ca84 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.458259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dbaef-b17b-4cb0-8ca8-e8edbce6ca84 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.458584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dbaef-b17b-4cb0-8ca8-e8edbce6ca84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.473642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:59.480882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-937dbaef-b17b-4cb0-8ca8-e8edbce6ca84 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:59.482239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-937dbaef-b17b-4cb0-8ca8-e8edbce6ca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:59.499137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1021/2043] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:59.503180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cad64-54b6-4ad6-9a03-cfe9a4c16088 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.513480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cad64-54b6-4ad6-9a03-cfe9a4c16088 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:59.518086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:59.518745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.519030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.519307 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.520072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:59.520387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:59.521972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:59.535883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1023/2044] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:59.538338 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dbaef-b17b-4cb0-8ca8-e8edbce6ca84 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.547690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dbaef-b17b-4cb0-8ca8-e8edbce6ca84 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:59.552484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:59.553217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.553602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.553951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.554726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:59.557253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:59.584318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1022/2045] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:24:59 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:59.588427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.598656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cacae-8472-4c05-8ec1-d47f40dbf1e0 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:59.658184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:59.658861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.659144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.659420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.660155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:59.662060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:59.688586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1024/2046] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:24:59 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:59.693985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.708429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6345-d55f-4479-a509-f5f6f5242c7e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:59.714733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:59.714733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.714733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.714733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.715120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:59.716721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:59.731136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1023/2047] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:59.735227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.745896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e302a-18ab-4811-a91c-f200f576837c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:59.750428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fd540-2967-4636-8b9e-afe988667640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:59.751079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fd540-2967-4636-8b9e-afe988667640 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.751364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fd540-2967-4636-8b9e-afe988667640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.751641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fd540-2967-4636-8b9e-afe988667640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.752441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fd540-2967-4636-8b9e-afe988667640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:59.754033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fd540-2967-4636-8b9e-afe988667640 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:59.763841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1025/2048] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:59.767778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.777821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca18f3c-4882-48d9-af10-4238f88997c8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:59.782821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:59.783750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.784092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.784359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] PATH_INFO: `/v3/projects/88edb7c1914f4c73ac0d4b10ac87bda4/users/0dd173405b7b441289356732575b109c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.799803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88edb7c1914f4c73ac0d4b10ac87bda4, user_id=0dd173405b7b441289356732575b109c, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:59.802506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:59.815787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1024/2049] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:24:59 2022] PUT /identity/v3/projects/88edb7c1914f4c73ac0d4b10ac87bda4/users/0dd173405b7b441289356732575b109c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:59.825701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:59.825997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:59.826283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:59.839856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:59.855287 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] MFA Rules not processed for user `0dd173405b7b441289356732575b109c`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:59.856543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:59.926750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1026/2050] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:24:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:59.938274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:59.940086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:59.997902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35451bd-c90e-47c4-8675-1e57f34599d9 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:00.001510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:00.002157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:00.002444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:00.002723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:00.011370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:00.012850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:00.023062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1025/2051] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:24:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:00.150921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:00.152573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:00.167023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343420e-9edf-434e-bb62-ce13f529bf11 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:00.170857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3382d-1b41-4882-b9d1-fbd85cb3e6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:00.171514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3382d-1b41-4882-b9d1-fbd85cb3e6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:00.171804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3382d-1b41-4882-b9d1-fbd85cb3e6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:00.172133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3382d-1b41-4882-b9d1-fbd85cb3e6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:00.174523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-10e3382d-1b41-4882-b9d1-fbd85cb3e6a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:00.252828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3382d-1b41-4882-b9d1-fbd85cb3e6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:00.256477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3382d-1b41-4882-b9d1-fbd85cb3e6a4 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:00.267088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1027/2052] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:02.182567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:02.184131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:02.269474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0de1-a77d-4c01-a513-bf9f60011f6b service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:02.274597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:02.279558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:02.279558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:02.279558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] PATH_INFO: `/v3/users/77a5a4342bab40108cd89fc1fde3a1cb` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:02.279558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a5a4342bab40108cd89fc1fde3a1cb)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:02.279558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:02.323954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a5a4342bab40108cd89fc1fde3a1cb'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.324589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a5a4342bab40108cd89fc1fde3a1cb'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.325184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a5a4342bab40108cd89fc1fde3a1cb'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.326275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:02.340237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a5a4342bab40108cd89fc1fde3a1cb'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.347064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a5a4342bab40108cd89fc1fde3a1cb'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.363685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1026/2053] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:02 2022] DELETE /identity/v3/users/77a5a4342bab40108cd89fc1fde3a1cb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:02.368482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:02.368938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:02.369047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:02.391945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:02.393154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:02.454989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1028/2054] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:02.465027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:02.466424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:02.521451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8903322-4805-4ba3-ab50-c85cef7d0920 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:02.525642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:02.526277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:02.526563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:02.526835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:02.535663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:02.537116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:02.546880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1027/2055] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:02.822086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:02.823916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:02.839245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb405ff0-06cb-47c4-b518-7e4288567a5d None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:02.845088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:02.846111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:02.846414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:02.846704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] PATH_INFO: `/v3/projects/7c04bae858fd49d1b060d30e8000f1b7` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:02.848043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c04bae858fd49d1b060d30e8000f1b7)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:02.857232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:02.892627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:02.913537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c04bae858fd49d1b060d30e8000f1b7', 'request_id': 'req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.921521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c04bae858fd49d1b060d30e8000f1b7', 'request_id': 'req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.944247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c04bae858fd49d1b060d30e8000f1b7', 'request_id': 'req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.956504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] The token cache is being invalidate because project 7c04bae858fd49d1b060d30e8000f1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:02.956873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.957619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:02.959659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1029/2056] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:02 2022] DELETE /identity/v3/projects/7c04bae858fd49d1b060d30e8000f1b7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:02.963660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:02.973802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bc711-7aff-45f8-bd7c-05b0bcbab19a service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:03.038024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:03.038902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:03.039890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:03.040176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] PATH_INFO: `/v3/users/200e6cb0657c449b93d08c4f9f4e8114` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:03.041379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=200e6cb0657c449b93d08c4f9f4e8114)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:03.043327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:03.088559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200e6cb0657c449b93d08c4f9f4e8114'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.089203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200e6cb0657c449b93d08c4f9f4e8114'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.089808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200e6cb0657c449b93d08c4f9f4e8114'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.102454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200e6cb0657c449b93d08c4f9f4e8114'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.106864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:03.110082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200e6cb0657c449b93d08c4f9f4e8114'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.124855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1028/2057] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:02 2022] DELETE /identity/v3/users/200e6cb0657c449b93d08c4f9f4e8114 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:03.489371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:03.491608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:03.513865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f9ee7-d533-41f4-8a7d-5a0c43e241d0 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:03.567554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:03.568438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:03.568720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:03.568995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] PATH_INFO: `/v3/projects/34b29b2cb9ab496fb9d179e22d8c8c05` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:03.570160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b29b2cb9ab496fb9d179e22d8c8c05)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:03.572622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:03.616358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b29b2cb9ab496fb9d179e22d8c8c05', 'request_id': 'req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.622533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b29b2cb9ab496fb9d179e22d8c8c05', 'request_id': 'req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.628130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b29b2cb9ab496fb9d179e22d8c8c05', 'request_id': 'req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.640566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] The token cache is being invalidate because project 34b29b2cb9ab496fb9d179e22d8c8c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:03.640841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.641469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.643024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1030/2058] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:03 2022] DELETE /identity/v3/projects/34b29b2cb9ab496fb9d179e22d8c8c05 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:03.647203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:03.649269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:03.661667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce977-260b-4c1f-9337-e700c94d0aeb admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:03.717033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:03.717901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:03.718185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:03.718467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] PATH_INFO: `/v3/users/fd73dfb4003d4247866793470be6674f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:03.719649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd73dfb4003d4247866793470be6674f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:03.721604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:03.766853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd73dfb4003d4247866793470be6674f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.767508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd73dfb4003d4247866793470be6674f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.768134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd73dfb4003d4247866793470be6674f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.780624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd73dfb4003d4247866793470be6674f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.787490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-afe94966-f280-4924-83cf-6afce43e6810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd73dfb4003d4247866793470be6674f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:03.800102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1029/2059] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:03 2022] DELETE /identity/v3/users/fd73dfb4003d4247866793470be6674f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:04.074351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.076027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:04.090911 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbdcdb-0ef3-4dd4-a465-e82386b34b26 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.134525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:04.150260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.151120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.151406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.151683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] PATH_INFO: `/v3/projects/2faafaaa37434aafb256f353fa512c10` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.152899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2faafaaa37434aafb256f353fa512c10)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.154794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.200055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2faafaaa37434aafb256f353fa512c10', 'request_id': 'req-45a91e39-ba52-4ad1-9371-1cde99c1a493'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:04.206282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2faafaaa37434aafb256f353fa512c10', 'request_id': 'req-45a91e39-ba52-4ad1-9371-1cde99c1a493'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:04.211784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2faafaaa37434aafb256f353fa512c10', 'request_id': 'req-45a91e39-ba52-4ad1-9371-1cde99c1a493'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:04.223385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] The token cache is being invalidate because project 2faafaaa37434aafb256f353fa512c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:04.223672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:04.224337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-45a91e39-ba52-4ad1-9371-1cde99c1a493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:04.225845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1031/2060] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:04 2022] DELETE /identity/v3/projects/2faafaaa37434aafb256f353fa512c10 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:04.232265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307847c-4258-4188-87b5-58538e57e72a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.232560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307847c-4258-4188-87b5-58538e57e72a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.232835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307847c-4258-4188-87b5-58538e57e72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.250383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b307847c-4258-4188-87b5-58538e57e72a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:04.251562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b307847c-4258-4188-87b5-58538e57e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:04.309955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1030/2061] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:04.319203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671163e-a8b2-4468-aebe-7f9fbdfb4250 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.319495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671163e-a8b2-4468-aebe-7f9fbdfb4250 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.319802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671163e-a8b2-4468-aebe-7f9fbdfb4250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.338706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4671163e-a8b2-4468-aebe-7f9fbdfb4250 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:04.339937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4671163e-a8b2-4468-aebe-7f9fbdfb4250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:04.349216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1032/2062] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:04.353911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307847c-4258-4188-87b5-58538e57e72a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.363471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307847c-4258-4188-87b5-58538e57e72a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.369113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.369603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.369888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.370162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.370894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:04.371208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.372850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.374472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:04.387463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1031/2063] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:04.392082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671163e-a8b2-4468-aebe-7f9fbdfb4250 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.401961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671163e-a8b2-4468-aebe-7f9fbdfb4250 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.407128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.407795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.408117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.408479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.409196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.411081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.438311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1033/2064] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:04 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:04.442452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.452501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df11d9c-555c-40c4-bd81-a98bd104c884 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.512610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.513316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.513601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.513883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.514600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.516675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.540763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1032/2065] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:04 2022] POST /identity/v3/users => generated 281 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:04.541977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.552308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552fd4f-aad7-4e78-9839-58c540d52d50 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.556905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.557571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.557861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.558145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.558883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.560527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.572624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1034/2066] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:04.576335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.587501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e250ef-8cdf-4a25-adba-dd5adb5e43db admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.595464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c28e2a-b950-4e1a-909c-4d77048f2102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.596282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28e2a-b950-4e1a-909c-4d77048f2102 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.596631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28e2a-b950-4e1a-909c-4d77048f2102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.596974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28e2a-b950-4e1a-909c-4d77048f2102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.597773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c28e2a-b950-4e1a-909c-4d77048f2102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.599433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c28e2a-b950-4e1a-909c-4d77048f2102 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.610706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1033/2067] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:04.614991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.625731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2706a-ee2b-4f58-9cf1-e521ade59fdd admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.630189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.631058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.631341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.631740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] PATH_INFO: `/v3/projects/e672e8a2f8994c67b3b20fc4b62645b5/users/6fcb60d2b4824dc2b332a796bc9688e4/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.647621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e672e8a2f8994c67b3b20fc4b62645b5, user_id=6fcb60d2b4824dc2b332a796bc9688e4, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.650354 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.666744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1035/2068] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:04 2022] PUT /identity/v3/projects/e672e8a2f8994c67b3b20fc4b62645b5/users/6fcb60d2b4824dc2b332a796bc9688e4/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:04.677643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc39b3-5d8a-48c1-9a5d-35004710ff4a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.678121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc39b3-5d8a-48c1-9a5d-35004710ff4a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.678228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc39b3-5d8a-48c1-9a5d-35004710ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.705157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-96bc39b3-5d8a-48c1-9a5d-35004710ff4a None None] MFA Rules not processed for user `6fcb60d2b4824dc2b332a796bc9688e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:04.706347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-96bc39b3-5d8a-48c1-9a5d-35004710ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:04.768164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1034/2069] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:04.798550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.800665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:04.856818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d8398-0ddc-4666-940f-7cc3d281666c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.860372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e4b6a-14dc-410c-b02d-ff18f2f6815b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.861006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e4b6a-14dc-410c-b02d-ff18f2f6815b service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.861287 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e4b6a-14dc-410c-b02d-ff18f2f6815b service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.861563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e4b6a-14dc-410c-b02d-ff18f2f6815b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.870187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e4b6a-14dc-410c-b02d-ff18f2f6815b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.871599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e4b6a-14dc-410c-b02d-ff18f2f6815b service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.880111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1036/2070] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:04.894378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc39b3-5d8a-48c1-9a5d-35004710ff4a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:04.955656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc39b3-5d8a-48c1-9a5d-35004710ff4a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:04.960225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:04.961136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:04.961417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:04.961687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] PATH_INFO: `/v3/users/6fcb60d2b4824dc2b332a796bc9688e4` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:04.962980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fcb60d2b4824dc2b332a796bc9688e4)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:04.964967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:04.967182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:05.017343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fcb60d2b4824dc2b332a796bc9688e4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.018000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fcb60d2b4824dc2b332a796bc9688e4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.018615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fcb60d2b4824dc2b332a796bc9688e4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.030037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fcb60d2b4824dc2b332a796bc9688e4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.036705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fcb60d2b4824dc2b332a796bc9688e4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.049494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1035/2071] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:04 2022] DELETE /identity/v3/users/6fcb60d2b4824dc2b332a796bc9688e4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:05.054132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.054549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.055425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.072281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:05.073474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:05.081847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:05.136725 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1037/2072] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:05.146327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:05.147683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:05.204077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c717443-290d-4639-bdae-182696283e70 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:05.208168 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0aa1ac-f999-4f3a-996f-a16ec331cbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:05.208825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa1ac-f999-4f3a-996f-a16ec331cbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.209110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa1ac-f999-4f3a-996f-a16ec331cbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.209390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa1ac-f999-4f3a-996f-a16ec331cbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.219004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0aa1ac-f999-4f3a-996f-a16ec331cbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:05.220384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0aa1ac-f999-4f3a-996f-a16ec331cbf2 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:05.230582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1036/2073] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:05.496027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:05.496027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:05.508840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dc02f-a48c-49d4-b749-6cc794f4551c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:05.514008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:05.514873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.515184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.515460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] PATH_INFO: `/v3/projects/e672e8a2f8994c67b3b20fc4b62645b5` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.516894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e672e8a2f8994c67b3b20fc4b62645b5)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:05.518991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:05.582282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e672e8a2f8994c67b3b20fc4b62645b5', 'request_id': 'req-41532b55-93b9-40a6-8911-086a94ca0f44'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.590182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e672e8a2f8994c67b3b20fc4b62645b5', 'request_id': 'req-41532b55-93b9-40a6-8911-086a94ca0f44'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.596300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e672e8a2f8994c67b3b20fc4b62645b5', 'request_id': 'req-41532b55-93b9-40a6-8911-086a94ca0f44'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.609352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] The token cache is being invalidate because project e672e8a2f8994c67b3b20fc4b62645b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:05.609641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.610271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-41532b55-93b9-40a6-8911-086a94ca0f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:05.611845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1038/2074] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:05 2022] DELETE /identity/v3/projects/e672e8a2f8994c67b3b20fc4b62645b5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:05.616819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9b67a-3388-4ef8-b71a-7df383f024c4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.617342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9b67a-3388-4ef8-b71a-7df383f024c4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.617723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9b67a-3388-4ef8-b71a-7df383f024c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.635670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fda9b67a-3388-4ef8-b71a-7df383f024c4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:05.637126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fda9b67a-3388-4ef8-b71a-7df383f024c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:05.697450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1037/2075] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:05.705690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8f93-c103-4dd8-af80-50a313191d6d None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.705983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8f93-c103-4dd8-af80-50a313191d6d None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.706263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8f93-c103-4dd8-af80-50a313191d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.726174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9b6a8f93-c103-4dd8-af80-50a313191d6d None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:05.727485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9b6a8f93-c103-4dd8-af80-50a313191d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:05.735766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1039/2076] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:05.740558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9b67a-3388-4ef8-b71a-7df383f024c4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:05.750106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9b67a-3388-4ef8-b71a-7df383f024c4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:05.754984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:05.755651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.755970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.756252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.756994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:05.757305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:05.759449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:05.771450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1038/2077] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:05.776771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8f93-c103-4dd8-af80-50a313191d6d None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:05.787194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8f93-c103-4dd8-af80-50a313191d6d None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:05.793880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:05.794206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.794549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.794838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.795672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:05.797948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:05.830922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1040/2078] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:05 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:05.841687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:05.848178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a24c5-06a9-4334-b315-f3792010eb20 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:05.858360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:05.913439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:05.914214 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.914502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.914785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.915521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:05.917514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:05.941628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1039/2079] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:05 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:05.942769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:05.953557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85dd0c-a7b5-494b-924c-ee4bfba39b7d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:05.958415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:05.959068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.959355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.959636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.960403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:05.962615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:05.974137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1041/2080] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:05.978182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:05.991102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154350fc-21a1-4c3b-9ea2-747cf15c96bd admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:05.996238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73306c8-9f35-486e-a0c9-dff9723e0b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:05.997091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73306c8-9f35-486e-a0c9-dff9723e0b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:05.997424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73306c8-9f35-486e-a0c9-dff9723e0b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:05.997755 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73306c8-9f35-486e-a0c9-dff9723e0b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:05.998570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73306c8-9f35-486e-a0c9-dff9723e0b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:06.000242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73306c8-9f35-486e-a0c9-dff9723e0b2d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:06.015377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1040/2081] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:06.019450 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:06.030065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b5283-5603-4215-89f9-17f445633900 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:06.034899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:06.036003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:06.036292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:06.036573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] PATH_INFO: `/v3/projects/6f76fe83a9c649c3ac4bc41a9bc60fd1/users/fc88bee433df4fa29033b72ba67c6f46/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:06.051771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f76fe83a9c649c3ac4bc41a9bc60fd1, user_id=fc88bee433df4fa29033b72ba67c6f46, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:06.054461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:06.067505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1042/2082] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:06 2022] PUT /identity/v3/projects/6f76fe83a9c649c3ac4bc41a9bc60fd1/users/fc88bee433df4fa29033b72ba67c6f46/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:06.079064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713eee-e00a-4360-a193-e497f6d7c3c2 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:06.079355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713eee-e00a-4360-a193-e497f6d7c3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:06.079921 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713eee-e00a-4360-a193-e497f6d7c3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:06.107221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ee713eee-e00a-4360-a193-e497f6d7c3c2 None None] MFA Rules not processed for user `fc88bee433df4fa29033b72ba67c6f46`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:06.108455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ee713eee-e00a-4360-a193-e497f6d7c3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:06.173986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1041/2083] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:06.186362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:06.188586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:06.231010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:06.260384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44ab44-ed3b-4bcf-82be-bbb72fef641b admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:06.264317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3e65-a955-448b-a2ea-8ed7d4fe9100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:06.265052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3e65-a955-448b-a2ea-8ed7d4fe9100 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:06.265462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3e65-a955-448b-a2ea-8ed7d4fe9100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:06.265747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3e65-a955-448b-a2ea-8ed7d4fe9100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:06.276332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3e65-a955-448b-a2ea-8ed7d4fe9100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:06.278053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3e65-a955-448b-a2ea-8ed7d4fe9100 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:06.290349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1043/2084] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:06.318571 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee713eee-e00a-4360-a193-e497f6d7c3c2 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:06.388530 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee713eee-e00a-4360-a193-e497f6d7c3c2 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:06.393175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:06.394075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:06.394354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:06.394624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] PATH_INFO: `/v3/users/fc88bee433df4fa29033b72ba67c6f46` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:06.395845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc88bee433df4fa29033b72ba67c6f46)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:06.397777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:06.420049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:06.445765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc88bee433df4fa29033b72ba67c6f46'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:06.446443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc88bee433df4fa29033b72ba67c6f46'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:06.447134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc88bee433df4fa29033b72ba67c6f46'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:06.460276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc88bee433df4fa29033b72ba67c6f46'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:06.467904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc88bee433df4fa29033b72ba67c6f46'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:06.481159 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1042/2085] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:06 2022] DELETE /identity/v3/users/fc88bee433df4fa29033b72ba67c6f46 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:06.485794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:06.486086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:06.486390 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:06.504391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:06.505601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:06.575736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1044/2086] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:06.589300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:06.591013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:06.660260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867dde7-2bf8-4ca3-bd4e-a60797590b7a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:06.666502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa19d78-a441-40fb-a3cb-b23444aac0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:06.667660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa19d78-a441-40fb-a3cb-b23444aac0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:06.668069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa19d78-a441-40fb-a3cb-b23444aac0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:06.668454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa19d78-a441-40fb-a3cb-b23444aac0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:06.685514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa19d78-a441-40fb-a3cb-b23444aac0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:06.687176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa19d78-a441-40fb-a3cb-b23444aac0e7 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:06.698759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1043/2087] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:06.973921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:06.975663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:06.988600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727418bd-3327-44fc-90c4-5abeda0b7199 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:06.993940 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:06.994849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:06.995140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:06.995419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] PATH_INFO: `/v3/users/d1d27f19405a4d3980910f783cbb8d16` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:06.996644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d27f19405a4d3980910f783cbb8d16)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:06.998567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:07.056696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d27f19405a4d3980910f783cbb8d16'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.057355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d27f19405a4d3980910f783cbb8d16'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.065295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d27f19405a4d3980910f783cbb8d16'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.066088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d27f19405a4d3980910f783cbb8d16'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.078786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d27f19405a4d3980910f783cbb8d16'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.096260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1045/2088] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:06 2022] DELETE /identity/v3/users/d1d27f19405a4d3980910f783cbb8d16 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:07.101139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:07.101447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:07.101724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:07.107130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:07.108735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:07.120490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:07.121687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:07.162798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:07.180520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f7538-165c-436b-bf50-0434e64660a9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:07.185149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1044/2089] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:07.185490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:07.186377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:07.186659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:07.186934 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] PATH_INFO: `/v3/projects/6f76fe83a9c649c3ac4bc41a9bc60fd1` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:07.188253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f76fe83a9c649c3ac4bc41a9bc60fd1)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:07.190164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:07.194808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:07.196268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:07.240920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f76fe83a9c649c3ac4bc41a9bc60fd1', 'request_id': 'req-0dc78cb6-25fb-4c84-8d82-97281a969384'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.245562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:07.247861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f76fe83a9c649c3ac4bc41a9bc60fd1', 'request_id': 'req-0dc78cb6-25fb-4c84-8d82-97281a969384'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.255126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f76fe83a9c649c3ac4bc41a9bc60fd1', 'request_id': 'req-0dc78cb6-25fb-4c84-8d82-97281a969384'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.261812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296ef30-d51d-420c-a80b-4460846841ee None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:07.268537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] The token cache is being invalidate because project 6f76fe83a9c649c3ac4bc41a9bc60fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:07.268826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.269462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.271222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1046/2090] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:07 2022] DELETE /identity/v3/projects/6f76fe83a9c649c3ac4bc41a9bc60fd1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:07.316593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:07.317261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:07.317542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:07.317820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:07.319734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:07.393695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:07.395335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:07.411796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1045/2091] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:07.679344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:07.680764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:07.693545 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc78cb6-25fb-4c84-8d82-97281a969384 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:07.699127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:07.700031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:07.700315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:07.700590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] PATH_INFO: `/v3/projects/e8cc4fcc64d3439e8fb3ab94927a37b1` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:07.701775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8cc4fcc64d3439e8fb3ab94927a37b1)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:07.703796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:07.752187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8cc4fcc64d3439e8fb3ab94927a37b1', 'request_id': 'req-edd96a78-6fc7-4422-8820-b735fd4b3eb3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.758308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8cc4fcc64d3439e8fb3ab94927a37b1', 'request_id': 'req-edd96a78-6fc7-4422-8820-b735fd4b3eb3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.763840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8cc4fcc64d3439e8fb3ab94927a37b1', 'request_id': 'req-edd96a78-6fc7-4422-8820-b735fd4b3eb3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.776387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] The token cache is being invalidate because project e8cc4fcc64d3439e8fb3ab94927a37b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:07.776678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.777296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.778820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1047/2092] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:07 2022] DELETE /identity/v3/projects/e8cc4fcc64d3439e8fb3ab94927a37b1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:07.782760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:07.793492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860c2d8-fdb6-4675-8fef-da4a1b02611f service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:07.848926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:07.849786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:07.850065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:07.850335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] PATH_INFO: `/v3/users/f3a5d4ca8b264dbf91e4eb9f71a94a3c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:07.852060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a5d4ca8b264dbf91e4eb9f71a94a3c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:07.853968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:07.898258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a5d4ca8b264dbf91e4eb9f71a94a3c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.900464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a5d4ca8b264dbf91e4eb9f71a94a3c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.900464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a5d4ca8b264dbf91e4eb9f71a94a3c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.902670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:07.914586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a5d4ca8b264dbf91e4eb9f71a94a3c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.921751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a91c2cd-8a1c-44fb-9986-acd1571322c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a5d4ca8b264dbf91e4eb9f71a94a3c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:07.939457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1046/2093] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:07 2022] DELETE /identity/v3/users/f3a5d4ca8b264dbf91e4eb9f71a94a3c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:08.300070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:08.301497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:08.316803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd96a78-6fc7-4422-8820-b735fd4b3eb3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:08.374118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:08.375013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.375304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.375584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] PATH_INFO: `/v3/projects/0920e0cdd10f4c329aaf770df6b2ee2b` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.376858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0920e0cdd10f4c329aaf770df6b2ee2b)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:08.378777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:08.381037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:08.429898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0920e0cdd10f4c329aaf770df6b2ee2b', 'request_id': 'req-6b43eb30-7816-42da-bb9a-d74682e56b15'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:08.436048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0920e0cdd10f4c329aaf770df6b2ee2b', 'request_id': 'req-6b43eb30-7816-42da-bb9a-d74682e56b15'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:08.441464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0920e0cdd10f4c329aaf770df6b2ee2b', 'request_id': 'req-6b43eb30-7816-42da-bb9a-d74682e56b15'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:08.455537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] The token cache is being invalidate because project 0920e0cdd10f4c329aaf770df6b2ee2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:08.455846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:08.456505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6b43eb30-7816-42da-bb9a-d74682e56b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:08.458078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1048/2094] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:08 2022] DELETE /identity/v3/projects/0920e0cdd10f4c329aaf770df6b2ee2b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:08.462902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1db2d-1bd9-4550-b704-b40b8b0b7c2b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.463193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1db2d-1bd9-4550-b704-b40b8b0b7c2b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.463468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1db2d-1bd9-4550-b704-b40b8b0b7c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.481380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e1e1db2d-1bd9-4550-b704-b40b8b0b7c2b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:08.482602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e1e1db2d-1bd9-4550-b704-b40b8b0b7c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:08.541627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1047/2095] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:08.549480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463fdf3-f3e1-4280-a5db-083b475fb0ca None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.549775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463fdf3-f3e1-4280-a5db-083b475fb0ca None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.550065 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463fdf3-f3e1-4280-a5db-083b475fb0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.569734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c463fdf3-f3e1-4280-a5db-083b475fb0ca None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:08.571580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c463fdf3-f3e1-4280-a5db-083b475fb0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:08.579831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1049/2096] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:08.584218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1db2d-1bd9-4550-b704-b40b8b0b7c2b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:08.593784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1db2d-1bd9-4550-b704-b40b8b0b7c2b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:08.598396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:08.599046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.599330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.599610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.600379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:08.600690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:08.602299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:08.613651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1048/2097] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:08.618466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463fdf3-f3e1-4280-a5db-083b475fb0ca None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:08.627651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463fdf3-f3e1-4280-a5db-083b475fb0ca None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:08.632359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:08.633555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.633842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.634122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.634826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:08.636794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:08.661054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1050/2098] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:08 2022] POST /identity/v3/projects => generated 393 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:08.665467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:08.675092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2ba34-5e74-4340-ae82-0aa276c2a130 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:08.730322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:08.730918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.731208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.731486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.732220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:08.734141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:08.753674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1049/2099] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:08 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:08.757833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:08.767814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688097f-d168-4f1e-bfa0-209f5187fc54 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:08.772433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:08.773087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.773374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.773656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.774396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:08.776096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:08.786254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1051/2100] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:08.790642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:08.801779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc5738-858a-4ae8-bd0d-ea05f9e9584b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:08.807095 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a0c82-3e3d-448a-beb6-faf13a93c547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:08.807748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a0c82-3e3d-448a-beb6-faf13a93c547 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.808068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a0c82-3e3d-448a-beb6-faf13a93c547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.808351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a0c82-3e3d-448a-beb6-faf13a93c547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.809087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a0c82-3e3d-448a-beb6-faf13a93c547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:08.810725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a0c82-3e3d-448a-beb6-faf13a93c547 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:08.821118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1050/2101] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:08.825319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:08.835445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fa3ce-39df-4c88-a412-58a7a577317d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:08.840605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:08.841478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.841774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.842054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] PATH_INFO: `/v3/projects/4e9a2b759e9f4931a6dc5679eb94d6cd/users/ff8a738113ab4f3cad705855af4307bc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.856267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e9a2b759e9f4931a6dc5679eb94d6cd, user_id=ff8a738113ab4f3cad705855af4307bc, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:08.858924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:08.872046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1052/2102] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:08 2022] PUT /identity/v3/projects/4e9a2b759e9f4931a6dc5679eb94d6cd/users/ff8a738113ab4f3cad705855af4307bc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:08.882183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70570ae-5959-474e-b02a-8cea75b12592 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:08.883010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70570ae-5959-474e-b02a-8cea75b12592 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:08.883298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70570ae-5959-474e-b02a-8cea75b12592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:08.910328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b70570ae-5959-474e-b02a-8cea75b12592 None None] MFA Rules not processed for user `ff8a738113ab4f3cad705855af4307bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:08.911577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b70570ae-5959-474e-b02a-8cea75b12592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:08.931403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:08.977936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1051/2103] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:08.982443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.042918 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a751f9-fd14-4d95-ac58-b90af3488790 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.047641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.048319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.048605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.048887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.049583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.051481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.077653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1053/2104] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:08 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:09.082848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70570ae-5959-474e-b02a-8cea75b12592 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.097338 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70570ae-5959-474e-b02a-8cea75b12592 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.159920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.160604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.160959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.161292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.162074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.164227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.210986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1052/2105] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:09 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:09.215126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.225853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23aeb-01c0-44b9-b435-94ecf4e97f17 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.230731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.231396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.231687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.232003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.232804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.234350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.246237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1054/2106] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:09.250358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.261223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1edd01-4023-4c10-a146-9b7e1ce414db admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.265923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.266805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.267083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.267354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] PATH_INFO: `/v3/projects/c381e5f4696143658c5e8d8be03f0bb0/users/50cd98294f0141f19c0e85de18c7b063/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.282152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c381e5f4696143658c5e8d8be03f0bb0, user_id=50cd98294f0141f19c0e85de18c7b063, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.285032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.299148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1053/2107] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:09 2022] PUT /identity/v3/projects/c381e5f4696143658c5e8d8be03f0bb0/users/50cd98294f0141f19c0e85de18c7b063/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:09.303261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.309125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:09.316000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1887b7e-6d8f-4a1b-b37c-f697dd6e2d90 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.372113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa3724-c4c2-4a8b-987b-9b89a123d75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.372779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3724-c4c2-4a8b-987b-9b89a123d75c admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.373068 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3724-c4c2-4a8b-987b-9b89a123d75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.373348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3724-c4c2-4a8b-987b-9b89a123d75c admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.374080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa3724-c4c2-4a8b-987b-9b89a123d75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.375705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa3724-c4c2-4a8b-987b-9b89a123d75c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.385825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1055/2108] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:09.389924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.400004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202944d-384c-47ed-81da-cb00a65bb16e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.404486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.405356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.405637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.405913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] PATH_INFO: `/v3/projects/c381e5f4696143658c5e8d8be03f0bb0/users/50cd98294f0141f19c0e85de18c7b063/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.407856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c381e5f4696143658c5e8d8be03f0bb0, user_id=50cd98294f0141f19c0e85de18c7b063, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.410528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.421625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1054/2109] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:09 2022] PUT /identity/v3/projects/c381e5f4696143658c5e8d8be03f0bb0/users/50cd98294f0141f19c0e85de18c7b063/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:09.433250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48710-aceb-4391-99a9-02b9384d8cc3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.433541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48710-aceb-4391-99a9-02b9384d8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.433844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48710-aceb-4391-99a9-02b9384d8cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.468022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-46d48710-aceb-4391-99a9-02b9384d8cc3 None None] MFA Rules not processed for user `50cd98294f0141f19c0e85de18c7b063`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:09.469225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-46d48710-aceb-4391-99a9-02b9384d8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:09.542598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1056/2110] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:09.552615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.554294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:09.609793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ad8d6-3b15-404e-bf3b-7cc73067e2c3 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.613419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.614043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.614324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.614596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.623779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.625224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.634295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1055/2111] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:09.680691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48710-aceb-4391-99a9-02b9384d8cc3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:09.690951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48710-aceb-4391-99a9-02b9384d8cc3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:09.694749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:09.695479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:09.695832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:09.696134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:09.758601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:09.759985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:09.769163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1057/2112] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:15.246178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:15.288686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:15.344716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77eef1-0bd0-4aae-9610-2a263e268449 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:15.350289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:15.351205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:15.351611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:15.351926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] PATH_INFO: `/v3/users/ff8a738113ab4f3cad705855af4307bc` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:15.353279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8a738113ab4f3cad705855af4307bc)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:15.355364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:15.410714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8a738113ab4f3cad705855af4307bc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.411322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8a738113ab4f3cad705855af4307bc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.412101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8a738113ab4f3cad705855af4307bc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.430282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8a738113ab4f3cad705855af4307bc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.447157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:15.449512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-dff51265-b5d7-44ce-8f92-8e062e74140b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8a738113ab4f3cad705855af4307bc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.450633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:15.466778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1056/2113] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:15 2022] DELETE /identity/v3/users/ff8a738113ab4f3cad705855af4307bc => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:15.467735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c7248-4809-49cb-9827-13ffded7604f service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:15.470985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:15.471275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:15.471553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:15.488495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:15.492868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:15.494325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:15.545575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:15.546462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:15.546899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:15.547184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] PATH_INFO: `/v3/users/e4d835d78d4f4b019ced356585af16c7` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:15.548434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d835d78d4f4b019ced356585af16c7)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:15.550370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:15.561483 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1057/2114] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:15.571864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:15.573205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:15.599109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d835d78d4f4b019ced356585af16c7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.599751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d835d78d4f4b019ced356585af16c7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.607625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d835d78d4f4b019ced356585af16c7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.608285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d835d78d4f4b019ced356585af16c7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.619082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:15.620689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d835d78d4f4b019ced356585af16c7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.635684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1058/2115] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:15 2022] DELETE /identity/v3/users/e4d835d78d4f4b019ced356585af16c7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:15.641176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e075d-b45f-4b30-b146-dcd1018019de None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:15.691926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecef576-b1ac-4752-bebe-48a1dcc7f713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:15.692584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef576-b1ac-4752-bebe-48a1dcc7f713 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:15.692871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef576-b1ac-4752-bebe-48a1dcc7f713 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:15.693151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef576-b1ac-4752-bebe-48a1dcc7f713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:15.695465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7ecef576-b1ac-4752-bebe-48a1dcc7f713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:15.759858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecef576-b1ac-4752-bebe-48a1dcc7f713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:15.761314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecef576-b1ac-4752-bebe-48a1dcc7f713 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:15.770867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1058/2116] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:15.852823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:15.854372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:15.884353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd59b7b-c23c-4304-b71d-8a126c6f7d49 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:15.889833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:15.890712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:15.891015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:15.891306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] PATH_INFO: `/v3/projects/de1626e5c02845f5b9b93e8b5fe916f0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:15.892551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1626e5c02845f5b9b93e8b5fe916f0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:15.895031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:15.961555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1626e5c02845f5b9b93e8b5fe916f0', 'request_id': 'req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.970082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1626e5c02845f5b9b93e8b5fe916f0', 'request_id': 'req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.978284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1626e5c02845f5b9b93e8b5fe916f0', 'request_id': 'req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.992015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] The token cache is being invalidate because project de1626e5c02845f5b9b93e8b5fe916f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:15.992519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.993150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:15.994919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1059/2117] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:15 2022] DELETE /identity/v3/projects/de1626e5c02845f5b9b93e8b5fe916f0 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:16.005269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d0af3-b92d-43be-b764-ed8fe0f0db03 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.005586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d0af3-b92d-43be-b764-ed8fe0f0db03 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.005872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d0af3-b92d-43be-b764-ed8fe0f0db03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.024149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-7b5d0af3-b92d-43be-b764-ed8fe0f0db03 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:16.025417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7b5d0af3-b92d-43be-b764-ed8fe0f0db03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.085943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.087472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.094364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1059/2118] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:16.100997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af549-b791-4c51-9f7a-1e662dc642b0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.101343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af549-b791-4c51-9f7a-1e662dc642b0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.101630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af549-b791-4c51-9f7a-1e662dc642b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.108022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bebab-94d3-4d16-b7cb-a8ea302d0aa2 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.113902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.114775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.115160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.115314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] PATH_INFO: `/v3/projects/4e9a2b759e9f4931a6dc5679eb94d6cd` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.117048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e9a2b759e9f4931a6dc5679eb94d6cd)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.118958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.122633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-368af549-b791-4c51-9f7a-1e662dc642b0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:16.123860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-368af549-b791-4c51-9f7a-1e662dc642b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.141847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:16.142725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1060/2119] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:16.147128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368af549-b791-4c51-9f7a-1e662dc642b0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.156746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368af549-b791-4c51-9f7a-1e662dc642b0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.161448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.162117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.162400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.162678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.163967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:16.164284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.165881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.172760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e9a2b759e9f4931a6dc5679eb94d6cd', 'request_id': 'req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.177834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1061/2120] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:25:16.179580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e9a2b759e9f4931a6dc5679eb94d6cd', 'request_id': 'req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.182642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.185850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e9a2b759e9f4931a6dc5679eb94d6cd', 'request_id': 'req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.192700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3c2bb-e5ce-4136-8a69-f66479a7b70a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.199173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] The token cache is being invalidate because project 4e9a2b759e9f4931a6dc5679eb94d6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:16.199563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.200226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.201804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1060/2121] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:16 2022] DELETE /identity/v3/projects/4e9a2b759e9f4931a6dc5679eb94d6cd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:16.205806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.207256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.214465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:16.252536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.253215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.253474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.253752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.254450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.256403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.272201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7f485-70b7-4b4c-84b3-3ce2603bf8cf admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.278016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.278895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.279334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.279644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] PATH_INFO: `/v3/users/50cd98294f0141f19c0e85de18c7b063` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.280892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cd98294f0141f19c0e85de18c7b063)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.282875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.285541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1062/2122] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:16.289776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.292252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.346051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cd98294f0141f19c0e85de18c7b063'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.346789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cd98294f0141f19c0e85de18c7b063'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.357265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cd98294f0141f19c0e85de18c7b063'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.357915 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cd98294f0141f19c0e85de18c7b063'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.376513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cd98294f0141f19c0e85de18c7b063'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.377888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705d135-ce69-4528-b13e-8cbfb86c202b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.383270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.384196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.384484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.384767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.385597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.387996 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.391680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1061/2123] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:16 2022] DELETE /identity/v3/users/50cd98294f0141f19c0e85de18c7b063 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:16.435737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1063/2124] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:16.438504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.440257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.482465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:16.536657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251358ae-c1df-40f6-a357-de8b5ed37316 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.545213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.546380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.546821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.548528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.550194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.552595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.573011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1062/2125] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:16.573560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.589650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e298a64-3f09-4f11-a360-af31cb302769 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.611580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8ab37-44a7-4297-b359-0fb325b4a1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.612480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ab37-44a7-4297-b359-0fb325b4a1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.614446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ab37-44a7-4297-b359-0fb325b4a1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.615420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ab37-44a7-4297-b359-0fb325b4a1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.617674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8ab37-44a7-4297-b359-0fb325b4a1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.619624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8ab37-44a7-4297-b359-0fb325b4a1ff admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.648989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.649223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1064/2126] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:16.661083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20236f1e-a820-4e3e-b537-3116b9375b0c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.667764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.668757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.669045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.669458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] PATH_INFO: `/v3/projects/356ed7112aba4a7ea825ed5e64f587a9/users/6dd482b4f21f4e42b58fff1bcdc3ad4d/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.691247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356ed7112aba4a7ea825ed5e64f587a9, user_id=6dd482b4f21f4e42b58fff1bcdc3ad4d, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.694356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.715290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1063/2127] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:16 2022] PUT /identity/v3/projects/356ed7112aba4a7ea825ed5e64f587a9/users/6dd482b4f21f4e42b58fff1bcdc3ad4d/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:16.728693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb82f09-8972-4da8-bf56-7a062ecab647 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.728977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb82f09-8972-4da8-bf56-7a062ecab647 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.729260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb82f09-8972-4da8-bf56-7a062ecab647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.760485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-dbb82f09-8972-4da8-bf56-7a062ecab647 None None] MFA Rules not processed for user `6dd482b4f21f4e42b58fff1bcdc3ad4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:16.761709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dbb82f09-8972-4da8-bf56-7a062ecab647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.764210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.766442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.858881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1065/2128] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:16.862148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb82f09-8972-4da8-bf56-7a062ecab647 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.866338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fecb-716e-45ac-a319-c883a8f35d26 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.871475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.872627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.872920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.873199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] PATH_INFO: `/v3/projects/c381e5f4696143658c5e8d8be03f0bb0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.874012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb82f09-8972-4da8-bf56-7a062ecab647 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:16.874526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c381e5f4696143658c5e8d8be03f0bb0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.876498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.879118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:16.880025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.880505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.880891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.881692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:16.884267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:16.899365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:16.916633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1066/2129] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:16.917731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:16.929574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c381e5f4696143658c5e8d8be03f0bb0', 'request_id': 'req-33993466-5378-4af5-aeaf-9dcedfd4c43f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.935779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c381e5f4696143658c5e8d8be03f0bb0', 'request_id': 'req-33993466-5378-4af5-aeaf-9dcedfd4c43f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.941647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c381e5f4696143658c5e8d8be03f0bb0', 'request_id': 'req-33993466-5378-4af5-aeaf-9dcedfd4c43f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.955275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] The token cache is being invalidate because project c381e5f4696143658c5e8d8be03f0bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:16.955497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.956187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-33993466-5378-4af5-aeaf-9dcedfd4c43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:16.957754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1064/2130] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:16 2022] DELETE /identity/v3/projects/c381e5f4696143658c5e8d8be03f0bb0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:16.963521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1194-3610-49c4-a974-b955761363da None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:16.963828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1194-3610-49c4-a974-b955761363da None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:16.964125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1194-3610-49c4-a974-b955761363da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:16.981874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-199a1194-3610-49c4-a974-b955761363da None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:16.983109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-199a1194-3610-49c4-a974-b955761363da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:16.989021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06c3a-df8d-4bac-905b-5f2c4023d44d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.007778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:17.046101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.046772 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.046923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1065/2131] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:17.048232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.048540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.049241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.051161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.054391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403158f-f133-47ef-9171-82e64360716c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.054810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403158f-f133-47ef-9171-82e64360716c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.055091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403158f-f133-47ef-9171-82e64360716c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.071508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1067/2132] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:16 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:17.073791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-e403158f-f133-47ef-9171-82e64360716c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:17.075222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e403158f-f133-47ef-9171-82e64360716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.075472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.077086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.083212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1066/2133] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:17.087671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403158f-f133-47ef-9171-82e64360716c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.092222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de8ee8-c0d1-440a-af21-1a46554b2c36 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.096286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403158f-f133-47ef-9171-82e64360716c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.097173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.097862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.098151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.098455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.099219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.101300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.101426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.101925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.102205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.102478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.103196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:17.103507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.105113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.111633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1068/2134] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:17.115734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.117778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1067/2135] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:25:17.121854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.126314 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260799-6391-4fab-9290-f5742f95fa02 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.130883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.131485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e30e94-db4e-4e4d-9aa2-8ff77722ec1e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.131746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.132090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.132367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] PATH_INFO: `/v3/projects/256147dbecbc48c28397a7fe5b849775/users/211ad96f9e144a999cdee38586e6a04e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.136160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.136780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.137069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.137363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.138052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.140048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.147185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256147dbecbc48c28397a7fe5b849775, user_id=211ad96f9e144a999cdee38586e6a04e, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.149890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.163669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1069/2136] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:17 2022] PUT /identity/v3/projects/256147dbecbc48c28397a7fe5b849775/users/211ad96f9e144a999cdee38586e6a04e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:17.167523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.168593 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1068/2137] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:17 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:17.172400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.226291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:17.230066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bf64d-f6ac-49aa-aa33-aaf40855c244 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.234690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c40c9-2b4b-482f-8e7b-9509eb19d57e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.236147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.237045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.237335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.237639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.238375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.239355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.240010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.240118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.240326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.240599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.241292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.243778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.250580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1070/2138] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:17.254787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.266492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1069/2139] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:17 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:17.266647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97310c-a7c6-4082-83fc-8b2fb67f2d36 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.267561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.269350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2822c59-5636-4217-92dd-200db5b3c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.270220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822c59-5636-4217-92dd-200db5b3c1b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.270508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822c59-5636-4217-92dd-200db5b3c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.270791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2822c59-5636-4217-92dd-200db5b3c1b5 admin admin] PATH_INFO: `/v3/projects/256147dbecbc48c28397a7fe5b849775/users/211ad96f9e144a999cdee38586e6a04e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.273355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2822c59-5636-4217-92dd-200db5b3c1b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256147dbecbc48c28397a7fe5b849775, user_id=211ad96f9e144a999cdee38586e6a04e, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.276062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2822c59-5636-4217-92dd-200db5b3c1b5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.277270 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d55ce2-d8fc-4356-8f93-d040c1bdf9de admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.281774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.282403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.282683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.282955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.283680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.285273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.286719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1071/2140] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:17 2022] PUT /identity/v3/projects/256147dbecbc48c28397a7fe5b849775/users/211ad96f9e144a999cdee38586e6a04e/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:17.296319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1070/2141] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:17.299078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.299369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.299647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.301178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.311211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a981de-7baf-4aad-8ca8-cde9e3e8842a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.326626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] MFA Rules not processed for user `211ad96f9e144a999cdee38586e6a04e`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:17.327846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.371369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.372089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.372374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.372652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.373381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.374998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.384884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1071/2142] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:17.388835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.393477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:17.399172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9f4fa-68b2-4086-8d30-bebcc8d748ed admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.405889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1072/2143] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:17.411039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181103bc-b53a-439a-aec4-8da86e8ab685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.412164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181103bc-b53a-439a-aec4-8da86e8ab685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.412451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181103bc-b53a-439a-aec4-8da86e8ab685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.412731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181103bc-b53a-439a-aec4-8da86e8ab685 admin admin] PATH_INFO: `/v3/projects/9bcb9946871b4514a42d4998c2eed5ee/users/b794e5491b73492f989b3fb7e24823c7/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.413885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.415730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.430677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181103bc-b53a-439a-aec4-8da86e8ab685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bcb9946871b4514a42d4998c2eed5ee, user_id=b794e5491b73492f989b3fb7e24823c7, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.433623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181103bc-b53a-439a-aec4-8da86e8ab685 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.448390 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1072/2144] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:17 2022] PUT /identity/v3/projects/9bcb9946871b4514a42d4998c2eed5ee/users/b794e5491b73492f989b3fb7e24823c7/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:17.460713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.461237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.461668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.482994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ddbab-5f69-418e-865c-d66dcf46f736 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.492238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] MFA Rules not processed for user `b794e5491b73492f989b3fb7e24823c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:17.493449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.537053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.537705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.537991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.538352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.566871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1073/2145] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:17.576473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:17.577455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.579179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.591695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044acaa-ce7d-4aac-a3bc-5aafc78ea30f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.595748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.596424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.596708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.596980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.605653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.605994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.607114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.607399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.616793 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1074/2146] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:17.617255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1073/2147] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:17.689949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:17.698961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190694aa-9489-42b5-afc8-995067fbb37b service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:17.702781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a13ace-0071-4e8a-8fbd-7a4c97e6726e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:17.703418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13ace-0071-4e8a-8fbd-7a4c97e6726e service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.703701 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13ace-0071-4e8a-8fbd-7a4c97e6726e service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.704257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13ace-0071-4e8a-8fbd-7a4c97e6726e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.706369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e4a13ace-0071-4e8a-8fbd-7a4c97e6726e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.774881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a13ace-0071-4e8a-8fbd-7a4c97e6726e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:17.776936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a13ace-0071-4e8a-8fbd-7a4c97e6726e service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:17.787623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1075/2148] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:19.656841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:19.658242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:19.721269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e8798-aea0-465e-bbbb-a93db4b1cd3f service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:19.725663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:19.726527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:19.726814 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:19.727086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] PATH_INFO: `/v3/users/0dd173405b7b441289356732575b109c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:19.728288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd173405b7b441289356732575b109c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:19.730205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:19.774207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd173405b7b441289356732575b109c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:19.774842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd173405b7b441289356732575b109c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:19.775442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd173405b7b441289356732575b109c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:19.786881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd173405b7b441289356732575b109c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:19.793927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd173405b7b441289356732575b109c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:19.806531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1074/2149] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:19 2022] DELETE /identity/v3/users/0dd173405b7b441289356732575b109c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:19.811202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:19.811707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:19.811801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:19.829140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:19.830328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:19.875500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:19.888731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1076/2150] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:19.898603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:19.900200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:19.959060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed455d-bfc2-4ea4-b756-2ca01c06cd9f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:19.962668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:19.963302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:19.963584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:19.963890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:19.969608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:19.974121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:19.976061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:19.987162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1075/2151] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:20.305300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:20.306821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:20.322645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517499b5-4ad4-4b6b-8726-6f484a8770de None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:20.327295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:20.335346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:20.335641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:20.335958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] PATH_INFO: `/v3/projects/88edb7c1914f4c73ac0d4b10ac87bda4` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:20.337277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88edb7c1914f4c73ac0d4b10ac87bda4)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:20.339210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:20.392049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88edb7c1914f4c73ac0d4b10ac87bda4', 'request_id': 'req-67f3825a-b205-49c3-8f68-94f3b3d13175'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:20.398718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88edb7c1914f4c73ac0d4b10ac87bda4', 'request_id': 'req-67f3825a-b205-49c3-8f68-94f3b3d13175'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:20.404295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88edb7c1914f4c73ac0d4b10ac87bda4', 'request_id': 'req-67f3825a-b205-49c3-8f68-94f3b3d13175'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:20.416926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] The token cache is being invalidate because project 88edb7c1914f4c73ac0d4b10ac87bda4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:20.417216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:20.417842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-67f3825a-b205-49c3-8f68-94f3b3d13175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:20.419395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1077/2152] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:20 2022] DELETE /identity/v3/projects/88edb7c1914f4c73ac0d4b10ac87bda4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:22.714028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:22.715580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:22.729198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db995f2-b5fd-4e75-9962-8cbd284c4daa service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:22.790311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:22.791352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:22.791981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:22.792517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] PATH_INFO: `/v3/users/6dd482b4f21f4e42b58fff1bcdc3ad4d` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:22.793862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd482b4f21f4e42b58fff1bcdc3ad4d)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:22.796227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:22.843574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd482b4f21f4e42b58fff1bcdc3ad4d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:22.844323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd482b4f21f4e42b58fff1bcdc3ad4d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:22.844984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd482b4f21f4e42b58fff1bcdc3ad4d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:22.858904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd482b4f21f4e42b58fff1bcdc3ad4d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:22.866317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd482b4f21f4e42b58fff1bcdc3ad4d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:22.881339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1076/2153] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:22 2022] DELETE /identity/v3/users/6dd482b4f21f4e42b58fff1bcdc3ad4d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:22.886233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:22.886724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:22.886924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:22.905104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:22.906377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:22.980917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1078/2154] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:22.990319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:22.991725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:23.052837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5065514-c094-4099-83e6-ae4a490764db admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:23.056658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:23.057305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:23.057594 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:23.057868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:23.066860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:23.068326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:23.078355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1077/2155] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:23.350223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:23.351678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:23.365769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10354-19d4-42bf-9732-c7371684b1a7 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:23.372081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:23.372983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:23.373272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:23.373611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] PATH_INFO: `/v3/projects/356ed7112aba4a7ea825ed5e64f587a9` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:23.374876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] RBAC: Authorizing `identity:delete_project(project_id=356ed7112aba4a7ea825ed5e64f587a9)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:23.376818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:23.427625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356ed7112aba4a7ea825ed5e64f587a9', 'request_id': 'req-e530c340-7f3f-48b5-ade5-ea115665298d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.435095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356ed7112aba4a7ea825ed5e64f587a9', 'request_id': 'req-e530c340-7f3f-48b5-ade5-ea115665298d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.441286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356ed7112aba4a7ea825ed5e64f587a9', 'request_id': 'req-e530c340-7f3f-48b5-ade5-ea115665298d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.451374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:23.456520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] The token cache is being invalidate because project 356ed7112aba4a7ea825ed5e64f587a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:23.456703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.457506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.459634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1079/2156] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:23 2022] DELETE /identity/v3/projects/356ed7112aba4a7ea825ed5e64f587a9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:23.464312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:23.483646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c41da-b9f1-4eb5-9c44-36d4b1918dd3 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:23.495741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:23.548687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:23.549704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:23.550033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:23.550310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] PATH_INFO: `/v3/users/211ad96f9e144a999cdee38586e6a04e` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:23.553212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=211ad96f9e144a999cdee38586e6a04e)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:23.554444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:23.609046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211ad96f9e144a999cdee38586e6a04e'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.609701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211ad96f9e144a999cdee38586e6a04e'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.610335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211ad96f9e144a999cdee38586e6a04e'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.623654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211ad96f9e144a999cdee38586e6a04e'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.633544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74d9ac1c-012c-46d1-aa68-ac7c2f012040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211ad96f9e144a999cdee38586e6a04e'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:23.648638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1078/2157] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:23 2022] DELETE /identity/v3/users/211ad96f9e144a999cdee38586e6a04e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:24.022991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.024596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:24.042413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530c340-7f3f-48b5-ade5-ea115665298d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.105949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.106740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.107027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.107300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] PATH_INFO: `/v3/projects/256147dbecbc48c28397a7fe5b849775` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.108477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=256147dbecbc48c28397a7fe5b849775)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.110363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.155044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256147dbecbc48c28397a7fe5b849775', 'request_id': 'req-72872f4c-9939-4c65-98e4-96a77b6bb7cf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:24.161028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256147dbecbc48c28397a7fe5b849775', 'request_id': 'req-72872f4c-9939-4c65-98e4-96a77b6bb7cf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:24.166632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256147dbecbc48c28397a7fe5b849775', 'request_id': 'req-72872f4c-9939-4c65-98e4-96a77b6bb7cf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:24.179461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] The token cache is being invalidate because project 256147dbecbc48c28397a7fe5b849775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:24.179756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:24.180598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-72872f4c-9939-4c65-98e4-96a77b6bb7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:24.182328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1080/2158] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:24 2022] DELETE /identity/v3/projects/256147dbecbc48c28397a7fe5b849775 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:24.187842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac8409-fdbf-402b-b482-5a96e76ffc44 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.188353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac8409-fdbf-402b-b482-5a96e76ffc44 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.188451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac8409-fdbf-402b-b482-5a96e76ffc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.207049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-22ac8409-fdbf-402b-b482-5a96e76ffc44 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:24.208736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-22ac8409-fdbf-402b-b482-5a96e76ffc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:24.279255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1079/2159] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:24.281607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b23c-a87a-4784-8e39-1335640720f5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.281899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b23c-a87a-4784-8e39-1335640720f5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.282177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b23c-a87a-4784-8e39-1335640720f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.304222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4893b23c-a87a-4784-8e39-1335640720f5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:24.305742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4893b23c-a87a-4784-8e39-1335640720f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:24.316235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1081/2160] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:24.320952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac8409-fdbf-402b-b482-5a96e76ffc44 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.331241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac8409-fdbf-402b-b482-5a96e76ffc44 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.336089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.336760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.337045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.337327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.338072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:24.338384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.340029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.351765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1080/2161] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:24.356229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893b23c-a87a-4784-8e39-1335640720f5 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.372229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893b23c-a87a-4784-8e39-1335640720f5 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.377112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.377786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.378175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.378342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.379086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.381039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.411958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1082/2162] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:24 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:24.414384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.425152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e029f33-4269-4b55-8a4c-e55361f286bc admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.444265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:24.484451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.485143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.485427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.485704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.486410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.488367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.509883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1081/2163] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:24 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:24.515171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.525602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4595161-de92-4a2b-abd2-76872db567e1 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.530343 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.531001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.531283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.531558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.532322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.533958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.548005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1083/2164] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:24.553897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.563770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9144-f8a9-4063-82ad-f71f3467c691 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.568473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069f719-1ce0-4028-a36a-c2543e9f7620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.569170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069f719-1ce0-4028-a36a-c2543e9f7620 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.569460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069f719-1ce0-4028-a36a-c2543e9f7620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.569741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069f719-1ce0-4028-a36a-c2543e9f7620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.570476 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069f719-1ce0-4028-a36a-c2543e9f7620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.572119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069f719-1ce0-4028-a36a-c2543e9f7620 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.582343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1082/2165] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:24.586527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.596671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ebba9-8ae0-4184-84da-6e4b450e0603 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.601792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.602662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.602972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.603247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] PATH_INFO: `/v3/projects/94f83440488f4f36a400548cb8defbb3/users/321ff68373554c74954dd1ca6ab634f9/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.619089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f83440488f4f36a400548cb8defbb3, user_id=321ff68373554c74954dd1ca6ab634f9, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.621724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.635988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1084/2166] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:24 2022] PUT /identity/v3/projects/94f83440488f4f36a400548cb8defbb3/users/321ff68373554c74954dd1ca6ab634f9/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:24.647804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52b342-4b00-42e0-9cd0-7043f4203e77 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.648114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52b342-4b00-42e0-9cd0-7043f4203e77 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.648390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52b342-4b00-42e0-9cd0-7043f4203e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.675099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0d52b342-4b00-42e0-9cd0-7043f4203e77 None None] MFA Rules not processed for user `321ff68373554c74954dd1ca6ab634f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:24.676420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0d52b342-4b00-42e0-9cd0-7043f4203e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:24.740607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1083/2167] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:24.745786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.778339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:24.814965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680580b6-6533-4e10-9f8e-25f97cce6a0c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.819648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.820636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.820636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.820869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.821659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.823717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.855953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1085/2168] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:24 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:24.859070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52b342-4b00-42e0-9cd0-7043f4203e77 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.869749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52b342-4b00-42e0-9cd0-7043f4203e77 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.930848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.931788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.932171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.932371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.933516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.935111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.955551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1084/2169] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:24 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:24.960054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:24.970503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd1586-6dcd-425d-b6a2-37190f553145 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:24.976348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:24.977024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:24.977822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:24.977822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:24.978386 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:24.980014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:24.991502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1086/2170] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:24.995782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:25.006424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa70cd-e70e-4478-ba11-e2803bf2cee9 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:25.012971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:25.013867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:25.014153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:25.014866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] PATH_INFO: `/v3/projects/6d4ea4a9eb7b42dbbea87cbc672f36ed/users/7cc7d9cb131745a2accbdab3c3b3e7b1/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:25.030612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4ea4a9eb7b42dbbea87cbc672f36ed, user_id=7cc7d9cb131745a2accbdab3c3b3e7b1, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:25.033401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:25.037243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:25.049073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1085/2171] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:24 2022] PUT /identity/v3/projects/6d4ea4a9eb7b42dbbea87cbc672f36ed/users/7cc7d9cb131745a2accbdab3c3b3e7b1/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:25.053327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:25.063488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b4005-11ce-4159-97e5-0c9d2ed0de0c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:25.134961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb282af-e378-4e55-8eb2-6a274718b6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:25.135562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb282af-e378-4e55-8eb2-6a274718b6be admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:25.135876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb282af-e378-4e55-8eb2-6a274718b6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:25.136159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb282af-e378-4e55-8eb2-6a274718b6be admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:25.136898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb282af-e378-4e55-8eb2-6a274718b6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:25.138535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb282af-e378-4e55-8eb2-6a274718b6be admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:25.151294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1087/2172] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:25.156125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:25.167316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d124d59-b1a3-4b4d-89b0-937e52b07aa2 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:25.172585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:25.173477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:25.173768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:25.174236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] PATH_INFO: `/v3/projects/6d4ea4a9eb7b42dbbea87cbc672f36ed/users/7cc7d9cb131745a2accbdab3c3b3e7b1/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:25.176310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4ea4a9eb7b42dbbea87cbc672f36ed, user_id=7cc7d9cb131745a2accbdab3c3b3e7b1, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:25.179981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:25.195602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1086/2173] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:25 2022] PUT /identity/v3/projects/6d4ea4a9eb7b42dbbea87cbc672f36ed/users/7cc7d9cb131745a2accbdab3c3b3e7b1/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:25.210299 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:25.210611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:25.210888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:25.239235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] MFA Rules not processed for user `7cc7d9cb131745a2accbdab3c3b3e7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:25.240463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:25.318059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1088/2174] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:25.330009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:25.332160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:25.395394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4d04-7ed3-49fe-92ff-ddfaae098d66 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:25.399651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:25.400318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:25.400606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:25.400887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:25.410221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:25.411737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:25.421754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1087/2175] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:25.456265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:25.458125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:25.493316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:25.536213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343080c7-449d-4aa3-bf97-ae6391d74084 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:25.540789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:25.543933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:25.543933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:25.543933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] PATH_INFO: `/v3/users/c22c34bc59fc4f6cb9d4449beb250a77` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:25.543933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c22c34bc59fc4f6cb9d4449beb250a77)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:25.545390 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:25.594907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c22c34bc59fc4f6cb9d4449beb250a77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.595564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c22c34bc59fc4f6cb9d4449beb250a77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.603063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c22c34bc59fc4f6cb9d4449beb250a77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.603969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c22c34bc59fc4f6cb9d4449beb250a77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.618541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c22c34bc59fc4f6cb9d4449beb250a77'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.632038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1089/2176] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:25 2022] DELETE /identity/v3/users/c22c34bc59fc4f6cb9d4449beb250a77 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:25.804914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:25.820318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54f1f1-43e4-4fdc-8d5d-dfa769c7743c service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:25.885332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:25.886200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:25.886484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:25.886763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] PATH_INFO: `/v3/projects/569038bf676a460c9fa21558c61132c2` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:25.887994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=569038bf676a460c9fa21558c61132c2)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:25.889926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:25.949596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569038bf676a460c9fa21558c61132c2', 'request_id': 'req-51e9e19b-6b57-4be0-8ca3-10ee8e706413'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.958830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569038bf676a460c9fa21558c61132c2', 'request_id': 'req-51e9e19b-6b57-4be0-8ca3-10ee8e706413'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.964523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569038bf676a460c9fa21558c61132c2', 'request_id': 'req-51e9e19b-6b57-4be0-8ca3-10ee8e706413'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.976269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] The token cache is being invalidate because project 569038bf676a460c9fa21558c61132c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:25.976556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.977220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:25.979112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1088/2177] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:25 2022] DELETE /identity/v3/projects/569038bf676a460c9fa21558c61132c2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:25.982883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:25.984312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:25.997917 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14f6d-4d76-4489-b8eb-bb3698909b45 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.028421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.029895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.048103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:26.059662 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.060580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.060872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.061158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] PATH_INFO: `/v3/users/1b1c67071d86461c8c3820fdd2701316` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.062444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1c67071d86461c8c3820fdd2701316)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.064416 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.097799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e19b-6b57-4be0-8ca3-10ee8e706413 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.103191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.104098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.104384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.104658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] PATH_INFO: `/v3/users/321ff68373554c74954dd1ca6ab634f9` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.105851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=321ff68373554c74954dd1ca6ab634f9)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.107775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.109742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:26.112597 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1c67071d86461c8c3820fdd2701316'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.113302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1c67071d86461c8c3820fdd2701316'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.121761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1c67071d86461c8c3820fdd2701316'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.122537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1c67071d86461c8c3820fdd2701316'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.135478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8ffc9a37-85e1-4506-b067-9c6db3bd824e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1c67071d86461c8c3820fdd2701316'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.147952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1090/2178] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:25 2022] DELETE /identity/v3/users/1b1c67071d86461c8c3820fdd2701316 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:26.154273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321ff68373554c74954dd1ca6ab634f9'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.154927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321ff68373554c74954dd1ca6ab634f9'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.155543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321ff68373554c74954dd1ca6ab634f9'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.167742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321ff68373554c74954dd1ca6ab634f9'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.175073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321ff68373554c74954dd1ca6ab634f9'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.189150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1089/2179] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:26 2022] DELETE /identity/v3/users/321ff68373554c74954dd1ca6ab634f9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:26.193987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.196144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.196428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.214936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:26.216235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.242108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.244407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.289199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1091/2180] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:26.300109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.301507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.336063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f759c-9f64-4d10-a3d0-7ae4ef181aff admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.342686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.343574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.343968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.344192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] PATH_INFO: `/v3/users/b794e5491b73492f989b3fb7e24823c7` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.345434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b794e5491b73492f989b3fb7e24823c7)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.347560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.391561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7f87f-83d1-47c7-8cd0-fd0855bb1bb8 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.395717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.396490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.396842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.397692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.405786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:26.408220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.409886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.417730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b794e5491b73492f989b3fb7e24823c7'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.418818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b794e5491b73492f989b3fb7e24823c7'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.419880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b794e5491b73492f989b3fb7e24823c7'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.422577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1092/2181] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:26.423551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.425017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.432989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b794e5491b73492f989b3fb7e24823c7'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.438466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eaa55f-9e10-436b-8402-665ec90644f2 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.441455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-205dd9f3-cc20-4593-a93f-7f7abc8d41ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b794e5491b73492f989b3fb7e24823c7'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.443875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.446005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.447428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.447902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] PATH_INFO: `/v3/projects/a810ff297df4432a9e60abf985ce063a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.450066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a810ff297df4432a9e60abf985ce063a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.452476 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.466056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1090/2182] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:26 2022] DELETE /identity/v3/users/b794e5491b73492f989b3fb7e24823c7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:26.467460 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.467753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.468062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.487087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:26.488195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.519523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a810ff297df4432a9e60abf985ce063a', 'request_id': 'req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.527384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a810ff297df4432a9e60abf985ce063a', 'request_id': 'req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.533057 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:26.535170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a810ff297df4432a9e60abf985ce063a', 'request_id': 'req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.551892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] The token cache is being invalidate because project a810ff297df4432a9e60abf985ce063a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:26.552175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.552967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1bbce5d6-9709-45e4-96e9-d1bda9f4931a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.555059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1093/2183] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:26 2022] DELETE /identity/v3/projects/a810ff297df4432a9e60abf985ce063a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:26.559396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65138-0814-41d3-b39d-90a442934532 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.559704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65138-0814-41d3-b39d-90a442934532 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.560021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65138-0814-41d3-b39d-90a442934532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.566006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1091/2184] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:26.575556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.577118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.586728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-38e65138-0814-41d3-b39d-90a442934532 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:26.588390 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-38e65138-0814-41d3-b39d-90a442934532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.658208 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858be2e0-4532-48ca-876f-ac2e616d95a3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.664324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1094/2185] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:26.672641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd049d5e-73ad-40bd-b5d7-e776a495e0fb None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.673281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd049d5e-73ad-40bd-b5d7-e776a495e0fb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.674108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd049d5e-73ad-40bd-b5d7-e776a495e0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.695010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-bd049d5e-73ad-40bd-b5d7-e776a495e0fb None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:26.696217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bd049d5e-73ad-40bd-b5d7-e776a495e0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.705630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1095/2186] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:26.712373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd049d5e-73ad-40bd-b5d7-e776a495e0fb None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.721867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.722609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.722904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.723178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.723460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd049d5e-73ad-40bd-b5d7-e776a495e0fb None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.733980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.735447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.744651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.745457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.745839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.746473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.746598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1092/2187] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:26.747122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.748461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.748764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:26.749347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.751576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.761829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acbb17-d941-4e5b-b4fb-6051354f1ecf service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.767608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.768745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.768909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.769188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] PATH_INFO: `/v3/projects/94f83440488f4f36a400548cb8defbb3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.770807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f83440488f4f36a400548cb8defbb3)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.773312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.775334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1096/2188] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:25:26.779913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.782383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:26.790653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be474f9-223f-4d75-a8b2-698098d2a5f6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.796414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.797432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.797598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.797985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.798863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.801171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.832211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1097/2189] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:26 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:26.837623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.837890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f83440488f4f36a400548cb8defbb3', 'request_id': 'req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.846346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f83440488f4f36a400548cb8defbb3', 'request_id': 'req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.854825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f83440488f4f36a400548cb8defbb3', 'request_id': 'req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.866809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] The token cache is being invalidate because project 94f83440488f4f36a400548cb8defbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:26.867151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.868060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:26.870423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1093/2190] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:26 2022] DELETE /identity/v3/projects/94f83440488f4f36a400548cb8defbb3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:26.875623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.877596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.908868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473f64-d9b9-40a9-ac2c-fea3f5cc9de3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.925953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.926281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.926557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.926837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.927892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.929759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.952766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1098/2191] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:26 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:26.956834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:26.958337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:26.971861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b853af7-06ac-4d6f-9fe3-a56911c8f2bf admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.975031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dd559-1cdd-4fe6-b2ad-5efe537fef04 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:26.976925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.977590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.977877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.978152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.978886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.980718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.982248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306366dd-c278-49fd-923e-388140042149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:26.983903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306366dd-c278-49fd-923e-388140042149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:26.984414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306366dd-c278-49fd-923e-388140042149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:26.984910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306366dd-c278-49fd-923e-388140042149 admin admin] PATH_INFO: `/v3/users/7cc7d9cb131745a2accbdab3c3b3e7b1` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:26.986801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306366dd-c278-49fd-923e-388140042149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc7d9cb131745a2accbdab3c3b3e7b1)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:26.989014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306366dd-c278-49fd-923e-388140042149 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:26.991261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1099/2192] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:26.994943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.008973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18273fd-4edd-47d5-8562-45027507f6d6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.015977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.016526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.016829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.017110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.017966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.019668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.033864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1100/2193] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:27.040086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.048710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc7d9cb131745a2accbdab3c3b3e7b1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.049633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc7d9cb131745a2accbdab3c3b3e7b1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.052604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc7d9cb131745a2accbdab3c3b3e7b1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.054638 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb8408-9a60-4598-a315-b61776ca6abc admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.060235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4797d1-4cc2-4c0a-a0a2-02045b635c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.061260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4797d1-4cc2-4c0a-a0a2-02045b635c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.064203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4797d1-4cc2-4c0a-a0a2-02045b635c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.064773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4797d1-4cc2-4c0a-a0a2-02045b635c2f admin admin] PATH_INFO: `/v3/projects/360e06d6ce7243f789b71a7b0743fd9a/users/f4d317c9d45449d5a00c024243310620/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.066672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc7d9cb131745a2accbdab3c3b3e7b1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.075154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc7d9cb131745a2accbdab3c3b3e7b1'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.085753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:27.091461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4797d1-4cc2-4c0a-a0a2-02045b635c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=360e06d6ce7243f789b71a7b0743fd9a, user_id=f4d317c9d45449d5a00c024243310620, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.092133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1094/2194] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:26 2022] DELETE /identity/v3/users/7cc7d9cb131745a2accbdab3c3b3e7b1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:27.093059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.094421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.098588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4797d1-4cc2-4c0a-a0a2-02045b635c2f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.117679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1101/2195] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:27 2022] PUT /identity/v3/projects/360e06d6ce7243f789b71a7b0743fd9a/users/f4d317c9d45449d5a00c024243310620/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:27.127841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:27.129424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.129832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.130600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.164619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] MFA Rules not processed for user `f4d317c9d45449d5a00c024243310620`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:27.165988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.179291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306366dd-c278-49fd-923e-388140042149 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.248880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.250084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.250439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.250889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] PATH_INFO: `/v3/projects/9bcb9946871b4514a42d4998c2eed5ee` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.252197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bcb9946871b4514a42d4998c2eed5ee)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.255311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1102/2196] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:27.255605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.266229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.267858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.309619 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bcb9946871b4514a42d4998c2eed5ee', 'request_id': 'req-870e3608-3450-41ef-8fb6-045b201ec640'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.317690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bcb9946871b4514a42d4998c2eed5ee', 'request_id': 'req-870e3608-3450-41ef-8fb6-045b201ec640'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.325719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bcb9946871b4514a42d4998c2eed5ee', 'request_id': 'req-870e3608-3450-41ef-8fb6-045b201ec640'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.334472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b02cb9-0173-4c45-8563-187a76046581 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.338489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] The token cache is being invalidate because project 9bcb9946871b4514a42d4998c2eed5ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:27.338919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.340258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-870e3608-3450-41ef-8fb6-045b201ec640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.342074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1095/2197] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:27 2022] DELETE /identity/v3/projects/9bcb9946871b4514a42d4998c2eed5ee => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:27.346499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.346789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.347067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.368144 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:27.369377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.390649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e2c4b-a591-47a2-8264-2627b89fb01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.391341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e2c4b-a591-47a2-8264-2627b89fb01d service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.391629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e2c4b-a591-47a2-8264-2627b89fb01d service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.391955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e2c4b-a591-47a2-8264-2627b89fb01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.393864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9a6e2c4b-a591-47a2-8264-2627b89fb01d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.434666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:27.448814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:27.452623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1096/2198] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:27.453941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.455218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.462488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e2c4b-a591-47a2-8264-2627b89fb01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.463983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e2c4b-a591-47a2-8264-2627b89fb01d service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.467759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5e7db-e1cc-4284-8502-d7f3b9d34235 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.472761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.473721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.473909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.474025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1103/2199] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:27.474187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] PATH_INFO: `/v3/projects/6d4ea4a9eb7b42dbbea87cbc672f36ed` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.475864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826b2b-ea6b-4400-977f-408d76579156 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.475995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d4ea4a9eb7b42dbbea87cbc672f36ed)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.476258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826b2b-ea6b-4400-977f-408d76579156 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.476550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826b2b-ea6b-4400-977f-408d76579156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.477893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.495071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ea826b2b-ea6b-4400-977f-408d76579156 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:27.496345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ea826b2b-ea6b-4400-977f-408d76579156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.504966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1104/2200] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:27.509174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826b2b-ea6b-4400-977f-408d76579156 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.525056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826b2b-ea6b-4400-977f-408d76579156 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.544322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d4ea4a9eb7b42dbbea87cbc672f36ed', 'request_id': 'req-6f346b0a-466c-4501-b5cd-2206454b2b47'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.557145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d4ea4a9eb7b42dbbea87cbc672f36ed', 'request_id': 'req-6f346b0a-466c-4501-b5cd-2206454b2b47'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.573549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d4ea4a9eb7b42dbbea87cbc672f36ed', 'request_id': 'req-6f346b0a-466c-4501-b5cd-2206454b2b47'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.591905 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] The token cache is being invalidate because project 6d4ea4a9eb7b42dbbea87cbc672f36ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:27.592922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.593819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-6f346b0a-466c-4501-b5cd-2206454b2b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:27.595531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1097/2201] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:27 2022] DELETE /identity/v3/projects/6d4ea4a9eb7b42dbbea87cbc672f36ed => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:27.598896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.600875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f291ba-0555-4732-8500-1f40a0ef4b1c None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.601162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f291ba-0555-4732-8500-1f40a0ef4b1c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.601300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.601463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f291ba-0555-4732-8500-1f40a0ef4b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.601561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.601744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.602496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:27.602856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.605213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.624844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-d3f291ba-0555-4732-8500-1f40a0ef4b1c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:27.625088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1105/2202] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:25:27.626362 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d3f291ba-0555-4732-8500-1f40a0ef4b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.629326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.631240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.641910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1098/2203] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:27.643580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87e665-d18b-418a-ac97-4260330e2740 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.644068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87e665-d18b-418a-ac97-4260330e2740 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.644350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87e665-d18b-418a-ac97-4260330e2740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.644721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85e674-6dd3-4978-98d9-aa03a0b52af9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.651602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.652344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.652684 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.652961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.653670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.656071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.663271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ac87e665-d18b-418a-ac97-4260330e2740 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:27.664682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ac87e665-d18b-418a-ac97-4260330e2740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:27.675165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1099/2204] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:27.679804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87e665-d18b-418a-ac97-4260330e2740 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.689254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87e665-d18b-418a-ac97-4260330e2740 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.696106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.696936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.697322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.697736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.698497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:27.698953 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.700685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.701900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1106/2205] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:27.706290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.715894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1100/2206] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:25:27.719059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.730863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daad63-73bf-4698-9a81-c047fb0b21df admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.786688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0a45-e5f0-46d5-89b2-8fec0f7a0866 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.793613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.794540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.794982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.795514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.796608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.799426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.801229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.802030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.802388 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.802640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.804582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.806944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.829944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1107/2207] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:27.831114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.842823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1101/2208] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:27.846681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.849782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7332e-946e-4caf-8206-93b3f034b260 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.898328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:27.903761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:27.926371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.927080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.927369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.927506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536effb-7031-4bb4-9655-34dd46547350 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.927653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.928465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.930082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.932237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.932905 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.933190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.933517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.934171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.936291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.941637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1108/2209] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:27.945544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.957181 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22f8bc-52ea-4476-af80-368b1c362d18 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.958900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1102/2210] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:27 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:27.963201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.965461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.966152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.966676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.966989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.967727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.969363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.972917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e712de9-f295-437a-a13d-166490cb4e29 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:27.978385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:27.979110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:27.979473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:27.979970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:27.980674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:27.980796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1109/2211] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:27.982511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:27.985330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.992871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1103/2212] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:27.996402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:27.997823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c01270-e1bd-448f-aa06-1b5212308073 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.002574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71d0c3-6029-4a8f-afb7-9c86768c1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.003450 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71d0c3-6029-4a8f-afb7-9c86768c1cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.003737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71d0c3-6029-4a8f-afb7-9c86768c1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.004054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71d0c3-6029-4a8f-afb7-9c86768c1cc5 admin admin] PATH_INFO: `/v3/projects/fa819f9c7cbf405baa1bb988cc854173/users/ea9d48b6245c497c8cc6ddcd56230672/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.005856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7477f-f628-4f13-a789-5d3d4f585261 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.010484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.011117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.011405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.011688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.012444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.014026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.019782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71d0c3-6029-4a8f-afb7-9c86768c1cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa819f9c7cbf405baa1bb988cc854173, user_id=ea9d48b6245c497c8cc6ddcd56230672, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.023143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71d0c3-6029-4a8f-afb7-9c86768c1cc5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.025462 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1104/2213] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:28.029448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.039834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1110/2214] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:27 2022] PUT /identity/v3/projects/fa819f9c7cbf405baa1bb988cc854173/users/ea9d48b6245c497c8cc6ddcd56230672/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:28.041285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95017d82-dfe7-4de9-8ed1-0d93a432e2d2 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.049886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.050179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.050465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.075785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] MFA Rules not processed for user `ea9d48b6245c497c8cc6ddcd56230672`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:28.077054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:28.099340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deb107-1e8b-445e-94e9-f27fcf38e59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.100811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb107-1e8b-445e-94e9-f27fcf38e59b admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.101093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb107-1e8b-445e-94e9-f27fcf38e59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.101370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb107-1e8b-445e-94e9-f27fcf38e59b admin admin] PATH_INFO: `/v3/projects/fb59a5d9cb0049d1bcfa35c97ea01e28/users/17e940957fd3421a9ee471deaf0e76f5/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.115264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deb107-1e8b-445e-94e9-f27fcf38e59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb59a5d9cb0049d1bcfa35c97ea01e28, user_id=17e940957fd3421a9ee471deaf0e76f5, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.118092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deb107-1e8b-445e-94e9-f27fcf38e59b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.133026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1105/2215] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:28 2022] PUT /identity/v3/projects/fb59a5d9cb0049d1bcfa35c97ea01e28/users/17e940957fd3421a9ee471deaf0e76f5/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:28.144058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631d01-2753-404d-ad01-5252cfe3e170 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.144339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631d01-2753-404d-ad01-5252cfe3e170 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.144620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631d01-2753-404d-ad01-5252cfe3e170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.148276 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1111/2216] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:28.161638 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.163206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:28.176228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-a6631d01-2753-404d-ad01-5252cfe3e170 None None] MFA Rules not processed for user `17e940957fd3421a9ee471deaf0e76f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:28.177511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a6631d01-2753-404d-ad01-5252cfe3e170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:28.220756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b720-b395-48ad-ae18-6d6373c7c15e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.242684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1106/2217] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:28.247223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6631d01-2753-404d-ad01-5252cfe3e170 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.264693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:28.274726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.275556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.276325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.276325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.280768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:28.312096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6631d01-2753-404d-ad01-5252cfe3e170 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.318021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.319265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.319879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.319997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.320694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.322845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.336373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.337842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.346815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1112/2218] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:28.352238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1107/2219] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:28 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:28.352721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.362785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9022a07-6925-4eab-a0a7-2e8dbc9c828e service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.421826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.422620 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.422972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.423355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.424119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.426093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.449418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1113/2220] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:28 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:28.450578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.461662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852623c6-970e-4b10-aa3b-4071605a3928 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.466461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.467131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.467412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.467687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.468466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.470093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.480781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1108/2221] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:28.485058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.496040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f561-53aa-489f-8485-b34a521eab8e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.500999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.501825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.502105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.502380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] PATH_INFO: `/v3/projects/b726862b44be4ed5adc43ff216c2b800/users/af1652ff9e7f431481923cb4c6152b6f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.519017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b726862b44be4ed5adc43ff216c2b800, user_id=af1652ff9e7f431481923cb4c6152b6f, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.522473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.538104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1114/2222] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:28 2022] PUT /identity/v3/projects/b726862b44be4ed5adc43ff216c2b800/users/af1652ff9e7f431481923cb4c6152b6f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:28.542360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.557129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb8801-0b6f-4db3-8b47-1ec20880b38f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.625106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a82a2e-39b7-4768-8a34-3ed0dbd8f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.625789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a82a2e-39b7-4768-8a34-3ed0dbd8f0dc admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.626077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a82a2e-39b7-4768-8a34-3ed0dbd8f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.626354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a82a2e-39b7-4768-8a34-3ed0dbd8f0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.627098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a82a2e-39b7-4768-8a34-3ed0dbd8f0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.629019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a82a2e-39b7-4768-8a34-3ed0dbd8f0dc admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.645176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1109/2223] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:28.646303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.656610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6c1e-5188-4e1f-956a-de51bfa53724 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.661274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.662154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.662472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.662747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] PATH_INFO: `/v3/projects/b726862b44be4ed5adc43ff216c2b800/users/af1652ff9e7f431481923cb4c6152b6f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.664829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b726862b44be4ed5adc43ff216c2b800, user_id=af1652ff9e7f431481923cb4c6152b6f, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.667669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.698387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1115/2224] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:28 2022] PUT /identity/v3/projects/b726862b44be4ed5adc43ff216c2b800/users/af1652ff9e7f431481923cb4c6152b6f/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:28.709454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.709750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.710024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.736517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] MFA Rules not processed for user `af1652ff9e7f431481923cb4c6152b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:28.737666 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:28.809527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1110/2225] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:28.824872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.879997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f6278-f741-40dc-a4e9-cb3163476208 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.884365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355a32c-f434-4426-85e5-14ba11ff11a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.885012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a32c-f434-4426-85e5-14ba11ff11a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.885306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a32c-f434-4426-85e5-14ba11ff11a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.885584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a32c-f434-4426-85e5-14ba11ff11a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.896003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:28.947193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355a32c-f434-4426-85e5-14ba11ff11a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.948673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355a32c-f434-4426-85e5-14ba11ff11a8 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.957845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1116/2226] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:30.601986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.603351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.616721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:30.666914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f815-b0b3-476d-a6a8-88c903627a76 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.671326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.672215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.672494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.672766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] PATH_INFO: `/v3/users/f4d317c9d45449d5a00c024243310620` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.673926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d317c9d45449d5a00c024243310620)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.675848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.721039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d317c9d45449d5a00c024243310620'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.721699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d317c9d45449d5a00c024243310620'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.722317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d317c9d45449d5a00c024243310620'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.734650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d317c9d45449d5a00c024243310620'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.742044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d317c9d45449d5a00c024243310620'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.754458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1111/2227] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:30 2022] DELETE /identity/v3/users/f4d317c9d45449d5a00c024243310620 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:30.758996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.759472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.759571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.776712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:30.777922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.842480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1117/2228] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:30.852751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.854183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.913125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92018472-0bbf-4351-8d52-58601d58e10e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.916908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeb562-9bde-49a3-892f-ac5b3f59e098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.917557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb562-9bde-49a3-892f-ac5b3f59e098 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.917842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb562-9bde-49a3-892f-ac5b3f59e098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.918149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb562-9bde-49a3-892f-ac5b3f59e098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.927763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeb562-9bde-49a3-892f-ac5b3f59e098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.929229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeb562-9bde-49a3-892f-ac5b3f59e098 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.938956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1112/2229] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:31.207935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.209872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:31.222384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb29a-92e3-4745-9e6b-1e871b1d83b6 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.227836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.228883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.229176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.229463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] PATH_INFO: `/v3/projects/360e06d6ce7243f789b71a7b0743fd9a` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.230709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] RBAC: Authorizing `identity:delete_project(project_id=360e06d6ce7243f789b71a7b0743fd9a)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.232688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.280757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360e06d6ce7243f789b71a7b0743fd9a', 'request_id': 'req-899e9292-aa56-4fa9-af06-83f0297f402d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.286893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360e06d6ce7243f789b71a7b0743fd9a', 'request_id': 'req-899e9292-aa56-4fa9-af06-83f0297f402d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.292639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360e06d6ce7243f789b71a7b0743fd9a', 'request_id': 'req-899e9292-aa56-4fa9-af06-83f0297f402d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.304787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] The token cache is being invalidate because project 360e06d6ce7243f789b71a7b0743fd9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:31.305064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.305705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-899e9292-aa56-4fa9-af06-83f0297f402d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.307305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1118/2230] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:31 2022] DELETE /identity/v3/projects/360e06d6ce7243f789b71a7b0743fd9a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:31.312279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef1aa-0179-4756-a16e-e43b386461f8 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.312572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef1aa-0179-4756-a16e-e43b386461f8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.312857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef1aa-0179-4756-a16e-e43b386461f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.331908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4c3ef1aa-0179-4756-a16e-e43b386461f8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:31.333100 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4c3ef1aa-0179-4756-a16e-e43b386461f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:31.363383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:31.395175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1113/2231] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:31.403558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d632-cbf6-46f5-9ef2-ca6dcaf067c7 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.403965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d632-cbf6-46f5-9ef2-ca6dcaf067c7 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.404255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d632-cbf6-46f5-9ef2-ca6dcaf067c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.423254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-5ef5d632-cbf6-46f5-9ef2-ca6dcaf067c7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:31.424485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5ef5d632-cbf6-46f5-9ef2-ca6dcaf067c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:31.432275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1119/2232] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:31.436346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ef1aa-0179-4756-a16e-e43b386461f8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.446508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ef1aa-0179-4756-a16e-e43b386461f8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.451022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.451669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.451995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.452274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.453005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:31.453319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.455169 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.467346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1114/2233] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:31.471423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5d632-cbf6-46f5-9ef2-ca6dcaf067c7 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.480348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5d632-cbf6-46f5-9ef2-ca6dcaf067c7 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.486570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.487213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.487509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.487964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.488755 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.490804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.520691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1120/2234] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:31.522724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.532791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917794-a130-4776-9b30-8d594d84beda admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.618926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.619575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.619918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.620198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.620990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.622916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.643424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1115/2235] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:31.647707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.657758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48bb50-8bc7-4992-adfd-691f2a72a1ab admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.662369 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.663023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.663307 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.663584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.664352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.665947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.667208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:31.678972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1121/2236] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:31.682607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.692422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d857e-e529-42bc-ab11-b39ca8cdfd74 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.697604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b5709-b5bc-4748-bcab-269a1ac54705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.698253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b5709-b5bc-4748-bcab-269a1ac54705 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.698540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b5709-b5bc-4748-bcab-269a1ac54705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.698820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b5709-b5bc-4748-bcab-269a1ac54705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.699546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b5709-b5bc-4748-bcab-269a1ac54705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.701180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b5709-b5bc-4748-bcab-269a1ac54705 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.711180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1116/2237] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:31.715140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.724980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e10c-46e6-4a68-9fa7-532b75d12a66 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.729599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.730546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.730845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.731183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] PATH_INFO: `/v3/projects/aee9b49e32b848e2a75facdeef1aa8a5/users/8b9bfcb74bc14e92b96e4169ed323399/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.745996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee9b49e32b848e2a75facdeef1aa8a5, user_id=8b9bfcb74bc14e92b96e4169ed323399, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.748692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.762025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1122/2238] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:31 2022] PUT /identity/v3/projects/aee9b49e32b848e2a75facdeef1aa8a5/users/8b9bfcb74bc14e92b96e4169ed323399/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:31.766914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069cbcf-c79d-4ce0-9c81-af2a5692c1ee None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.767233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069cbcf-c79d-4ce0-9c81-af2a5692c1ee None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.767514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069cbcf-c79d-4ce0-9c81-af2a5692c1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.786038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1069cbcf-c79d-4ce0-9c81-af2a5692c1ee None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:31.787231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1069cbcf-c79d-4ce0-9c81-af2a5692c1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:31.845274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1117/2239] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:31.854664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.856423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:31.914195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b760a-4ebe-4ee2-8aa0-2a7fdda9d0bd admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.917831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.918493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.918781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.919064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.929254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.930692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.942996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1123/2240] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:32.624697 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.625221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.625323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.652283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] MFA Rules not processed for user `8b9bfcb74bc14e92b96e4169ed323399`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:32.653473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:32.682618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:32.721089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1118/2241] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:32.731374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.732858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:32.796578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a5804-658c-4d0e-b0f8-7ae5b3cb3e0e service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.801894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45294f9-2c6f-410f-916d-61847de81c57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.802160 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45294f9-2c6f-410f-916d-61847de81c57 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.802317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45294f9-2c6f-410f-916d-61847de81c57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.802786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45294f9-2c6f-410f-916d-61847de81c57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.811244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45294f9-2c6f-410f-916d-61847de81c57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.812777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45294f9-2c6f-410f-916d-61847de81c57 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.822478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1124/2242] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:33.603062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:33.604470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:33.618054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca395fd-db2a-4ca9-81fa-7c12eb562aa6 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:33.622567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:33.623421 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:33.623730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:33.624034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] PATH_INFO: `/v3/users/17e940957fd3421a9ee471deaf0e76f5` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:33.625201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e940957fd3421a9ee471deaf0e76f5)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:33.627112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:33.672690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e940957fd3421a9ee471deaf0e76f5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:33.673339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e940957fd3421a9ee471deaf0e76f5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:33.673953 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e940957fd3421a9ee471deaf0e76f5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:33.686195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e940957fd3421a9ee471deaf0e76f5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:33.693043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e940957fd3421a9ee471deaf0e76f5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:33.705833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1119/2243] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:33 2022] DELETE /identity/v3/users/17e940957fd3421a9ee471deaf0e76f5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:33.710281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:33.710607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:33.710891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:33.728412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:33.729623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:33.787704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1125/2244] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:33.797863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:33.799297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:33.855513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6150e-2eb5-47f8-9abb-e86519544395 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:33.859158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:33.859794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:33.860118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:33.860398 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:33.869256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:33.870678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:33.879944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1120/2245] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:34.145608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:34.147681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:34.160866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f22f77-a2db-4598-a35e-3c14c58112a8 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:34.166332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:34.167236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:34.167529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:34.177667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] PATH_INFO: `/v3/projects/fb59a5d9cb0049d1bcfa35c97ea01e28` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:34.179444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb59a5d9cb0049d1bcfa35c97ea01e28)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:34.181474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:34.194618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:34.230029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb59a5d9cb0049d1bcfa35c97ea01e28', 'request_id': 'req-fa7f588d-1a8f-4eec-9e08-5195a77782fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.235995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb59a5d9cb0049d1bcfa35c97ea01e28', 'request_id': 'req-fa7f588d-1a8f-4eec-9e08-5195a77782fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.241617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb59a5d9cb0049d1bcfa35c97ea01e28', 'request_id': 'req-fa7f588d-1a8f-4eec-9e08-5195a77782fb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.253517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] The token cache is being invalidate because project fb59a5d9cb0049d1bcfa35c97ea01e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:34.253810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.254438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.256531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1126/2246] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:34 2022] DELETE /identity/v3/projects/fb59a5d9cb0049d1bcfa35c97ea01e28 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:34.260458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:34.269848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e09d-cef4-40bd-bb50-eafa709728e2 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:34.311440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:34.327327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:34.328218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:34.328759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:34.329027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] PATH_INFO: `/v3/users/af1652ff9e7f431481923cb4c6152b6f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:34.330202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1652ff9e7f431481923cb4c6152b6f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:34.332153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:34.376713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1652ff9e7f431481923cb4c6152b6f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.377377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1652ff9e7f431481923cb4c6152b6f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.377987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1652ff9e7f431481923cb4c6152b6f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.390107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1652ff9e7f431481923cb4c6152b6f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.397527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-696d53dc-1d6b-4f37-a8be-028f6ab13ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1652ff9e7f431481923cb4c6152b6f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.409774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1121/2247] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:34 2022] DELETE /identity/v3/users/af1652ff9e7f431481923cb4c6152b6f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:34.714010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:34.715418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:34.728642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f588d-1a8f-4eec-9e08-5195a77782fb admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:34.800470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:34.801338 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:34.801617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:34.801889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] PATH_INFO: `/v3/projects/b726862b44be4ed5adc43ff216c2b800` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:34.803585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b726862b44be4ed5adc43ff216c2b800)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:34.805520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:34.851187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b726862b44be4ed5adc43ff216c2b800', 'request_id': 'req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.857140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b726862b44be4ed5adc43ff216c2b800', 'request_id': 'req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.862497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b726862b44be4ed5adc43ff216c2b800', 'request_id': 'req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.875177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] The token cache is being invalidate because project b726862b44be4ed5adc43ff216c2b800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:34.875468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.876140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-226f56f2-c5cf-4db2-b85c-bb0924f0bc70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:34.877667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1127/2248] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:34 2022] DELETE /identity/v3/projects/b726862b44be4ed5adc43ff216c2b800 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:34.883547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941772-39ae-44dd-9f60-2dc6a8a19233 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:34.884039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941772-39ae-44dd-9f60-2dc6a8a19233 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:34.884128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941772-39ae-44dd-9f60-2dc6a8a19233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:34.901152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ee941772-39ae-44dd-9f60-2dc6a8a19233 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:34.902335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ee941772-39ae-44dd-9f60-2dc6a8a19233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:34.966192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1122/2249] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:34.973924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1d7a5-7b6e-432f-898b-671689b5d4ab None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:34.974260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1d7a5-7b6e-432f-898b-671689b5d4ab None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:34.975020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1d7a5-7b6e-432f-898b-671689b5d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:34.993897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-dfb1d7a5-7b6e-432f-898b-671689b5d4ab None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:34.995077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-dfb1d7a5-7b6e-432f-898b-671689b5d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:35.002776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1128/2250] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:35.007127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941772-39ae-44dd-9f60-2dc6a8a19233 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:35.016638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941772-39ae-44dd-9f60-2dc6a8a19233 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:35.021198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:35.021864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.022180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.022461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.023190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:35.023498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:35.025166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:35.036488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1123/2251] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:35.041216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1d7a5-7b6e-432f-898b-671689b5d4ab None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:35.050274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1d7a5-7b6e-432f-898b-671689b5d4ab None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:35.055422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:35.056101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.056389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.056669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.057364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:35.059264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:35.084077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1129/2252] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:35 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:35.088166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:35.097854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae634d0-df60-4b76-8218-db9d4166acab admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:35.143760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:35.154963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:35.155617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.155936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.156218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.156922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:35.159041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:35.179871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1124/2253] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:35 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:35.183575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:35.189690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:35.195731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc939-b0df-4d81-afde-021b0ca71f91 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:35.202801 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:35.204202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.205085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.205541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.206313 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:35.213387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:35.229289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1130/2254] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:35.230451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:35.241580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b19e8d-9d9d-4ffc-80e0-f0f28b4955cb admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:35.246152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54d484-1961-4e2c-a02f-84f5458da6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:35.246803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54d484-1961-4e2c-a02f-84f5458da6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.247090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54d484-1961-4e2c-a02f-84f5458da6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.247365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54d484-1961-4e2c-a02f-84f5458da6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.248132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54d484-1961-4e2c-a02f-84f5458da6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:35.249715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54d484-1961-4e2c-a02f-84f5458da6e4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:35.261291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1125/2255] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:35.265303 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:35.275554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e629-14cf-44f0-83c3-7dba83428c44 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:35.280110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:35.280982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.281282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.281559 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] PATH_INFO: `/v3/projects/2c0d374cce0245ebb63be064536eaaf5/users/9ea31ef824b3417abc5653a55c46e930/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.296520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0d374cce0245ebb63be064536eaaf5, user_id=9ea31ef824b3417abc5653a55c46e930, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:35.299204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:35.311736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1131/2256] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:35 2022] PUT /identity/v3/projects/2c0d374cce0245ebb63be064536eaaf5/users/9ea31ef824b3417abc5653a55c46e930/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:35.322948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244dcbc-4416-480b-8ace-c5cc11885d52 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.323254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244dcbc-4416-480b-8ace-c5cc11885d52 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.323529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244dcbc-4416-480b-8ace-c5cc11885d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.350254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b244dcbc-4416-480b-8ace-c5cc11885d52 None None] MFA Rules not processed for user `9ea31ef824b3417abc5653a55c46e930`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:35.351437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b244dcbc-4416-480b-8ace-c5cc11885d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:35.415519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1126/2257] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:35.427012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:35.428695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:35.485866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d071867-3fc6-4090-8ca0-b53dbee726d8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:35.489538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7be3a546aa644281a0d3594395bbdf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:35.490164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:35.490454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:35.490735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:35.498825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:35.500290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:35.508827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1132/2258] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:37.344305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127367a-39da-4164-9586-618fa45e85c3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:37.344916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127367a-39da-4164-9586-618fa45e85c3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:37.345075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127367a-39da-4164-9586-618fa45e85c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:37.363354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2127367a-39da-4164-9586-618fa45e85c3 None None] MFA Rules not processed for user `8b9bfcb74bc14e92b96e4169ed323399`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:37.364648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2127367a-39da-4164-9586-618fa45e85c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:37.424020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1127/2259] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:45.505227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:47.417606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:48.057031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:48.058419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:48.129148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8f6f-bb47-4e48-8805-3bf55f93d701 service glance] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:48.134572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:48.135629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:48.135952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:48.136237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] PATH_INFO: `/v3/users/ea9d48b6245c497c8cc6ddcd56230672` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:48.137452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9d48b6245c497c8cc6ddcd56230672)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:48.139504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:48.209944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9d48b6245c497c8cc6ddcd56230672'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.210886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9d48b6245c497c8cc6ddcd56230672'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.221346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9d48b6245c497c8cc6ddcd56230672'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.222060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9d48b6245c497c8cc6ddcd56230672'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.236178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9d48b6245c497c8cc6ddcd56230672'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.250211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1133/2260] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:48 2022] DELETE /identity/v3/users/ea9d48b6245c497c8cc6ddcd56230672 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:48.256326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:48.257053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:48.257773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:48.277709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:48.279470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:48.351475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1128/2261] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:48.361883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:48.363492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:48.426267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b38d-ac18-47a2-a515-cc8de36cddf0 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:48.431007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c0c81-4eea-4c26-bb18-383a3fc4a3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:48.432024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0c81-4eea-4c26-bb18-383a3fc4a3be service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:48.432528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0c81-4eea-4c26-bb18-383a3fc4a3be service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:48.432810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0c81-4eea-4c26-bb18-383a3fc4a3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:48.448061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c0c81-4eea-4c26-bb18-383a3fc4a3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:48.449569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c0c81-4eea-4c26-bb18-383a3fc4a3be service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:48.459960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1134/2262] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:48.833968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:48.835436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:48.848611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefe37c-5e64-4849-b3f1-c1e5bbf35106 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:48.854163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:48.855033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:48.855323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:48.855605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] PATH_INFO: `/v3/projects/fa819f9c7cbf405baa1bb988cc854173` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:48.856848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa819f9c7cbf405baa1bb988cc854173)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:48.858773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:48.908060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa819f9c7cbf405baa1bb988cc854173', 'request_id': 'req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.916586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa819f9c7cbf405baa1bb988cc854173', 'request_id': 'req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.925650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa819f9c7cbf405baa1bb988cc854173', 'request_id': 'req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.945881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] The token cache is being invalidate because project fa819f9c7cbf405baa1bb988cc854173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:48.946167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.946822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5b47ec8c-416f-4c1a-8564-e8d1e3afd066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:48.948409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1129/2263] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:48 2022] DELETE /identity/v3/projects/fa819f9c7cbf405baa1bb988cc854173 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:48.956256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93a321-2101-4866-971d-a118c05510cf None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:48.956256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93a321-2101-4866-971d-a118c05510cf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:48.956256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93a321-2101-4866-971d-a118c05510cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:48.972972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3e93a321-2101-4866-971d-a118c05510cf None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:48.974794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3e93a321-2101-4866-971d-a118c05510cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:48.991398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:49.035869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1135/2264] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:49.045122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e732-d653-462d-9297-aac0e80566bd None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.045418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e732-d653-462d-9297-aac0e80566bd None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.045702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e732-d653-462d-9297-aac0e80566bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.063709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-87a9e732-d653-462d-9297-aac0e80566bd None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:49.064919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-87a9e732-d653-462d-9297-aac0e80566bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:49.072700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1130/2265] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:49.076779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93a321-2101-4866-971d-a118c05510cf None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.087023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93a321-2101-4866-971d-a118c05510cf None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.091555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.092241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.092537 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.092800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.093524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:49.093856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.095440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.107370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1136/2266] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:49.111425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9e732-d653-462d-9297-aac0e80566bd None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.120424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9e732-d653-462d-9297-aac0e80566bd None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.125015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.125658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.125945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.126223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.126936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.128876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.155257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1131/2267] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:49.159355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.171569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f98b8-a7c2-4095-ad72-93942f5b4cc3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.229977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.230650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.230934 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.231214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.231945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.233862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.255647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1137/2268] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:49.259966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.269944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5843d7-c3c8-4e79-84b2-181a931fd5a8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.275027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.275683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.275988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.276266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.276988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.278579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.293532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1132/2269] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:49.294709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.307066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a8e8f-adc1-44c2-aab5-68bc6be2e5a8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.312133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bba209-86c4-4a28-9f63-d4117e3e38ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.312800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bba209-86c4-4a28-9f63-d4117e3e38ef admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.313083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bba209-86c4-4a28-9f63-d4117e3e38ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.313360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bba209-86c4-4a28-9f63-d4117e3e38ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.314095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bba209-86c4-4a28-9f63-d4117e3e38ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.315691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bba209-86c4-4a28-9f63-d4117e3e38ef admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.331293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1138/2270] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:49.332797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.344449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ee69-4252-45dc-9397-268a4c499153 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.349872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.351055 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.351431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.361374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] PATH_INFO: `/v3/projects/ecd9e0b988b94db0917be5fccebc6452/users/32ad4862b30649b19f763c9a812fe2ba/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.390145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd9e0b988b94db0917be5fccebc6452, user_id=32ad4862b30649b19f763c9a812fe2ba, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.393579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.410395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1133/2271] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:49 2022] PUT /identity/v3/projects/ecd9e0b988b94db0917be5fccebc6452/users/32ad4862b30649b19f763c9a812fe2ba/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:49.420465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acab7d7-a75b-4f5f-b705-9eedf1418af4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.421024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acab7d7-a75b-4f5f-b705-9eedf1418af4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.422058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acab7d7-a75b-4f5f-b705-9eedf1418af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.450078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7acab7d7-a75b-4f5f-b705-9eedf1418af4 None None] MFA Rules not processed for user `32ad4862b30649b19f763c9a812fe2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:49.451734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7acab7d7-a75b-4f5f-b705-9eedf1418af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:49.518784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1139/2272] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:49.524126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.547964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:49.605807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218704c-4bb4-4f18-acb8-23e3de370ef5 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.610416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.611076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.611360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.611637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.612367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.614789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.645662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1134/2273] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:49.649714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acab7d7-a75b-4f5f-b705-9eedf1418af4 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.660508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acab7d7-a75b-4f5f-b705-9eedf1418af4 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.673262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:49.723789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.724492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.724899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.725051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.725746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.727648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.755583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1140/2274] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:49.759389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.770228 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790000c-d6fd-4082-b23c-ba9a50c10e73 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.774901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.775561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.775865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.776172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.776900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.778491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.790025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1135/2275] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:49.794076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.807129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974624-da92-4ee4-836b-6a1943c98d76 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.812103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4d66b-615f-4b80-8284-8c7e08343ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.812842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4d66b-615f-4b80-8284-8c7e08343ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.813246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4d66b-615f-4b80-8284-8c7e08343ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.813530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4d66b-615f-4b80-8284-8c7e08343ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.814277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4d66b-615f-4b80-8284-8c7e08343ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.815970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4d66b-615f-4b80-8284-8c7e08343ed2 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.826884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1141/2276] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:49.831189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:49.840931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a267e-64de-4bc1-a682-efe8ecf1bd8d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:49.845535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:49.846409 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.846688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.846992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] PATH_INFO: `/v3/projects/d41750be9431495f83316072424ca84c/users/f3bee35cb8754fd58c8060394516bc90/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.861712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41750be9431495f83316072424ca84c, user_id=f3bee35cb8754fd58c8060394516bc90, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:49.864918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:49.881248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1136/2277] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:49 2022] PUT /identity/v3/projects/d41750be9431495f83316072424ca84c/users/f3bee35cb8754fd58c8060394516bc90/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:49.892335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3379cb-9cfa-4f9a-9a4f-d638ef7822cf None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:49.892626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3379cb-9cfa-4f9a-9a4f-d638ef7822cf None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:49.892906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3379cb-9cfa-4f9a-9a4f-d638ef7822cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:49.921660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-be3379cb-9cfa-4f9a-9a4f-d638ef7822cf None None] MFA Rules not processed for user `f3bee35cb8754fd58c8060394516bc90`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:49.927778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-be3379cb-9cfa-4f9a-9a4f-d638ef7822cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:49.996318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1142/2278] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:50.000893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:50.060598 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8bff7-0bfe-46ee-8f70-ddb4741cb1ee admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:50.065210 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:50.065859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.066149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.066428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.067124 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:50.069097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:50.093136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1137/2279] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:49 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:50.097418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3379cb-9cfa-4f9a-9a4f-d638ef7822cf None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:50.111533 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3379cb-9cfa-4f9a-9a4f-d638ef7822cf None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:50.168162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:50.168820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.169105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.169384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.170103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:50.172114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:50.191371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1143/2280] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:50 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:50.195576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:50.205849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f100d03-afb5-465d-a9ab-bdb81ed174cf admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:50.210432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:50.211109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.211395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.211672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.212447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:50.214038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:50.224908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1138/2281] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:50.228831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:50.238432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3766f-60c3-4b01-9f90-ff97af0b4540 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:50.246054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:50.246937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.247222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.247495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] PATH_INFO: `/v3/projects/5ccaafd6be964df49dcfe76a064840f6/users/eb00ccde95f149f7812f14f46765aafc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.261921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:50.265410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccaafd6be964df49dcfe76a064840f6, user_id=eb00ccde95f149f7812f14f46765aafc, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:50.268492 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:50.281734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1144/2282] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:50 2022] PUT /identity/v3/projects/5ccaafd6be964df49dcfe76a064840f6/users/eb00ccde95f149f7812f14f46765aafc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:50.285671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:50.297589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b0c73-b07b-4088-ac3b-bed8ffcdc5c6 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:50.341470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:50.359135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a70ab-9ad2-4372-9835-096780dbaf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:50.359813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a70ab-9ad2-4372-9835-096780dbaf93 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.360123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a70ab-9ad2-4372-9835-096780dbaf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.360400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a70ab-9ad2-4372-9835-096780dbaf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.361137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a70ab-9ad2-4372-9835-096780dbaf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:50.362752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a70ab-9ad2-4372-9835-096780dbaf93 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:50.373295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1139/2283] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:50.377626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:50.388258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b703dd0-2aa2-444f-94e4-6797ef2edc02 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:50.392947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:50.393827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.394116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.394400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] PATH_INFO: `/v3/projects/5ccaafd6be964df49dcfe76a064840f6/users/eb00ccde95f149f7812f14f46765aafc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.396895 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccaafd6be964df49dcfe76a064840f6, user_id=eb00ccde95f149f7812f14f46765aafc, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:50.399524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:50.409761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1145/2284] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:50 2022] PUT /identity/v3/projects/5ccaafd6be964df49dcfe76a064840f6/users/eb00ccde95f149f7812f14f46765aafc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:50.420701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fa6a-cd30-461a-b742-bbe36f74daf9 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.421008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fa6a-cd30-461a-b742-bbe36f74daf9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.421295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fa6a-cd30-461a-b742-bbe36f74daf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.446941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1eb4fa6a-cd30-461a-b742-bbe36f74daf9 None None] MFA Rules not processed for user `eb00ccde95f149f7812f14f46765aafc`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:50.448107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1eb4fa6a-cd30-461a-b742-bbe36f74daf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:50.521173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1140/2285] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:50.532798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:50.534473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:50.604879 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f6b82-9501-4396-8876-36ac4787f53f admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:50.610617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:50.611682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:50.612082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:50.612406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:50.677805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:50.679332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:50.689085 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1146/2286] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:52.252797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4fa6a-cd30-461a-b742-bbe36f74daf9 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:52.262330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4fa6a-cd30-461a-b742-bbe36f74daf9 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:52.266988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:52.267629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:52.267945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:52.268227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:52.326795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:52.328624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:52.337986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1141/2287] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:52.943026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:52.951689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b2d2c-f59f-4ffa-9055-bdbf67bb22ad service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:52.955657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481ee9-4fbc-41af-b3a6-d09b9a31fe95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:52.956292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481ee9-4fbc-41af-b3a6-d09b9a31fe95 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:52.956574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481ee9-4fbc-41af-b3a6-d09b9a31fe95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:52.956847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481ee9-4fbc-41af-b3a6-d09b9a31fe95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:52.966067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481ee9-4fbc-41af-b3a6-d09b9a31fe95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:52.967487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481ee9-4fbc-41af-b3a6-d09b9a31fe95 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:52.978921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1147/2288] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:56.882172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:56.967345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:57.012713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495ec47-817b-4324-80fd-6e4b71a52cc4 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:57.017206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:57.018082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:57.018360 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:57.018632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] PATH_INFO: `/v3/users/32ad4862b30649b19f763c9a812fe2ba` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:57.020042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ad4862b30649b19f763c9a812fe2ba)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:57.022587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:57.072659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ad4862b30649b19f763c9a812fe2ba'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.073315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ad4862b30649b19f763c9a812fe2ba'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.073930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ad4862b30649b19f763c9a812fe2ba'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.086073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ad4862b30649b19f763c9a812fe2ba'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.093463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ad4862b30649b19f763c9a812fe2ba'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.106590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1142/2289] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:56 2022] DELETE /identity/v3/users/32ad4862b30649b19f763c9a812fe2ba => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:57.111149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:57.111647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:57.111647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:57.162808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:57.164056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:57.217758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:57.232290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1148/2290] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:57.243601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:57.245099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:57.350848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd5e9-3f9a-4d66-99bc-cbd07b89b92a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:57.355132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:57.355794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:57.356151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:57.356474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:57.369370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:57.372613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:57.390914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1143/2291] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:57.767001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:57.768509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:57.780488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7e70c-fe2a-4c50-94d8-503fe687cead None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:57.786158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:57.787146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:57.787501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:57.787996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] PATH_INFO: `/v3/projects/ecd9e0b988b94db0917be5fccebc6452` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:57.789239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd9e0b988b94db0917be5fccebc6452)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:57.791165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:57.842121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd9e0b988b94db0917be5fccebc6452', 'request_id': 'req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.848794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd9e0b988b94db0917be5fccebc6452', 'request_id': 'req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.854362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd9e0b988b94db0917be5fccebc6452', 'request_id': 'req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.866664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] The token cache is being invalidate because project ecd9e0b988b94db0917be5fccebc6452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:57.866664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.867256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:57.868837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1149/2292] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:57 2022] DELETE /identity/v3/projects/ecd9e0b988b94db0917be5fccebc6452 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:57.873042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:57.885688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f565d51-e9e4-44fe-9aa0-bea0f0787ad3 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:57.891326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:57.962275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:57.963262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:57.963662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:57.963812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] PATH_INFO: `/v3/users/f3bee35cb8754fd58c8060394516bc90` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:57.965098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3bee35cb8754fd58c8060394516bc90)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:57.967015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.048316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3bee35cb8754fd58c8060394516bc90'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.049034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3bee35cb8754fd58c8060394516bc90'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.049760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3bee35cb8754fd58c8060394516bc90'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.062841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3bee35cb8754fd58c8060394516bc90'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.071042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3bee35cb8754fd58c8060394516bc90'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.086506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1144/2293] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:57 2022] DELETE /identity/v3/users/f3bee35cb8754fd58c8060394516bc90 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:58.361108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.363583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:58.377105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b0636-0a8f-46e1-bfc1-13af9533cfe3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.438697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.439562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.439885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.440168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] PATH_INFO: `/v3/projects/d41750be9431495f83316072424ca84c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.441349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41750be9431495f83316072424ca84c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.443797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.489575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41750be9431495f83316072424ca84c', 'request_id': 'req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.495565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41750be9431495f83316072424ca84c', 'request_id': 'req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.501439 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41750be9431495f83316072424ca84c', 'request_id': 'req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.514148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] The token cache is being invalidate because project d41750be9431495f83316072424ca84c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:58.514484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.515074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.516631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1150/2294] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:58 2022] DELETE /identity/v3/projects/d41750be9431495f83316072424ca84c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:58.521195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.522572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:58.535845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ea375-8190-4676-af8d-019222ccedd0 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.601790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.602668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.602951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.603229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] PATH_INFO: `/v3/users/eb00ccde95f149f7812f14f46765aafc` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.604531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb00ccde95f149f7812f14f46765aafc)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.606511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.640943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:58.656552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb00ccde95f149f7812f14f46765aafc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.657226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb00ccde95f149f7812f14f46765aafc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.657821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb00ccde95f149f7812f14f46765aafc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.669885 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb00ccde95f149f7812f14f46765aafc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.677255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-fb3a599f-0c8e-48cf-92b5-ad7fc214d0ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb00ccde95f149f7812f14f46765aafc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.690825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1145/2295] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:25:58 2022] DELETE /identity/v3/users/eb00ccde95f149f7812f14f46765aafc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:59.005711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.007237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:59.021461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3440-bee1-4b23-9a0a-d0ba4ed4bace admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.025864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:59.091315 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.092211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.092502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.092893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] PATH_INFO: `/v3/projects/5ccaafd6be964df49dcfe76a064840f6` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.094222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccaafd6be964df49dcfe76a064840f6)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.096341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.143008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccaafd6be964df49dcfe76a064840f6', 'request_id': 'req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:59.149434 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccaafd6be964df49dcfe76a064840f6', 'request_id': 'req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:59.155275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccaafd6be964df49dcfe76a064840f6', 'request_id': 'req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:59.168735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] The token cache is being invalidate because project 5ccaafd6be964df49dcfe76a064840f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:59.169055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:59.169683 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-1faa7dc5-dddb-4c55-b5ff-cab311f8600f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:59.175783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1151/2296] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:25:59 2022] DELETE /identity/v3/projects/5ccaafd6be964df49dcfe76a064840f6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:59.177411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e485bab-508c-47c2-b280-1e8db5287e6a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.177861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e485bab-508c-47c2-b280-1e8db5287e6a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.178137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e485bab-508c-47c2-b280-1e8db5287e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.216685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-3e485bab-508c-47c2-b280-1e8db5287e6a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:59.218044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3e485bab-508c-47c2-b280-1e8db5287e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:59.287654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1146/2297] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:59.295505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e084c-67a3-46b9-94b9-a2fd9dec1406 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.295856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e084c-67a3-46b9-94b9-a2fd9dec1406 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.296158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e084c-67a3-46b9-94b9-a2fd9dec1406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.315734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-8e1e084c-67a3-46b9-94b9-a2fd9dec1406 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:59.317543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8e1e084c-67a3-46b9-94b9-a2fd9dec1406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:59.325950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1152/2298] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:59.330239 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e485bab-508c-47c2-b280-1e8db5287e6a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.340331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e485bab-508c-47c2-b280-1e8db5287e6a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.344912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.345561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.345848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.346126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.346892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:59.347200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.348963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.361366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1147/2299] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:59.366130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e084c-67a3-46b9-94b9-a2fd9dec1406 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.375251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e084c-67a3-46b9-94b9-a2fd9dec1406 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.380664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.381347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.381634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.381912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.382614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.384618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.412557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1153/2300] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:25:59 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:59.415974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.426792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d86ac-3da1-4be7-a3ca-474f9b589160 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.484206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.484897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.485191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.485497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.486223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.488234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.509254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1148/2301] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:25:59 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:59.513335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.523648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701686-fc71-4217-9fdf-4c2c41a7c0b4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.528455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.529146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.529433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.529713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.530461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.532147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.543045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1154/2302] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:59.547181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.557569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99570b6-1c47-4f7a-a436-e352339dbd05 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.562204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104de9fd-aabf-4651-b9bf-feec73e62245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.562952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104de9fd-aabf-4651-b9bf-feec73e62245 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.563246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104de9fd-aabf-4651-b9bf-feec73e62245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.563490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104de9fd-aabf-4651-b9bf-feec73e62245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.564349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104de9fd-aabf-4651-b9bf-feec73e62245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.565999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104de9fd-aabf-4651-b9bf-feec73e62245 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.569034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:59.579511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1149/2303] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:59.583838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.593977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bb23a-9ed9-4872-991c-3875188fbce3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.598839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.599726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.600051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.600329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] PATH_INFO: `/v3/projects/f2868eb3201b4798bf6fe689fa7bf4f8/users/52350d51130345f38e9e43f686659d00/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.619367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2868eb3201b4798bf6fe689fa7bf4f8, user_id=52350d51130345f38e9e43f686659d00, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.623527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.638135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1155/2304] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:25:59 2022] PUT /identity/v3/projects/f2868eb3201b4798bf6fe689fa7bf4f8/users/52350d51130345f38e9e43f686659d00/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:59.651516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bba18-0602-4722-b4db-d808c7010435 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.651837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bba18-0602-4722-b4db-d808c7010435 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.652137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bba18-0602-4722-b4db-d808c7010435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.679858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ae1bba18-0602-4722-b4db-d808c7010435 None None] MFA Rules not processed for user `52350d51130345f38e9e43f686659d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:59.680990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ae1bba18-0602-4722-b4db-d808c7010435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:59.744032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1150/2305] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:25:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:59.755685 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.757480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:59.792948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:59.819331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72710858-d48b-4c10-b5e3-93f63da2c87a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.822989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2928f-8883-49ee-b926-de45499302c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.823652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2928f-8883-49ee-b926-de45499302c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.823966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2928f-8883-49ee-b926-de45499302c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.824250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2928f-8883-49ee-b926-de45499302c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.832909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2928f-8883-49ee-b926-de45499302c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.834337 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2928f-8883-49ee-b926-de45499302c4 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.842975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1156/2306] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:25:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:04.580492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bba18-0602-4722-b4db-d808c7010435 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:04.643396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bba18-0602-4722-b4db-d808c7010435 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:04.647954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:04.648819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:04.649102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:04.649379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] PATH_INFO: `/v3/users/52350d51130345f38e9e43f686659d00` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:04.650555 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52350d51130345f38e9e43f686659d00)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:04.652532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:04.706187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52350d51130345f38e9e43f686659d00'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:04.706834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52350d51130345f38e9e43f686659d00'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:04.707435 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52350d51130345f38e9e43f686659d00'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:04.719426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52350d51130345f38e9e43f686659d00'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:04.726310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52350d51130345f38e9e43f686659d00'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:04.742173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1151/2307] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:26:04 2022] DELETE /identity/v3/users/52350d51130345f38e9e43f686659d00 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:04.746619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:04.747115 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:04.747360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:04.767036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:04.768419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:04.835178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1157/2308] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:04.844675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:04.846246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:04.857555 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:04.907012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04b3d4-77c2-4cd4-a14b-c8637b566c58 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:04.910626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612da2a7-e9da-412a-9206-344091e34676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:04.911785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612da2a7-e9da-412a-9206-344091e34676 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:04.912102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612da2a7-e9da-412a-9206-344091e34676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:04.912385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612da2a7-e9da-412a-9206-344091e34676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:04.921633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612da2a7-e9da-412a-9206-344091e34676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:04.923077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612da2a7-e9da-412a-9206-344091e34676 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:04.933574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1152/2309] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:05.220241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.220241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:05.247170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bb2cc-e622-4fb2-af80-20ca9ab94bfe None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:05.251805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:05.252700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.252984 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.253379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] PATH_INFO: `/v3/projects/f2868eb3201b4798bf6fe689fa7bf4f8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.254921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2868eb3201b4798bf6fe689fa7bf4f8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:05.257269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:05.312566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2868eb3201b4798bf6fe689fa7bf4f8', 'request_id': 'req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:05.321266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2868eb3201b4798bf6fe689fa7bf4f8', 'request_id': 'req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:05.329771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2868eb3201b4798bf6fe689fa7bf4f8', 'request_id': 'req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:05.349688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] The token cache is being invalidate because project f2868eb3201b4798bf6fe689fa7bf4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:26:05.350501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:05.352636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e604d6d-ce8a-4791-823b-a4ad1c0da5e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:05.358867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1158/2310] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:26:05 2022] DELETE /identity/v3/projects/f2868eb3201b4798bf6fe689fa7bf4f8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:05.360611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeed15-5733-4f9a-a039-6166e0cd2b8e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.360908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeed15-5733-4f9a-a039-6166e0cd2b8e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.361181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeed15-5733-4f9a-a039-6166e0cd2b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.386026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-79aeed15-5733-4f9a-a039-6166e0cd2b8e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:05.387373 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-79aeed15-5733-4f9a-a039-6166e0cd2b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:05.453590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1153/2311] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:05.461756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db787-64ad-474f-8f9e-d4479f5e36e1 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.462048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db787-64ad-474f-8f9e-d4479f5e36e1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.462328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db787-64ad-474f-8f9e-d4479f5e36e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.485650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f67db787-64ad-474f-8f9e-d4479f5e36e1 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:05.486891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f67db787-64ad-474f-8f9e-d4479f5e36e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:05.495143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1159/2312] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:05.499433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeed15-5733-4f9a-a039-6166e0cd2b8e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.509775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeed15-5733-4f9a-a039-6166e0cd2b8e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:05.519528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:05.520776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.529867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.529867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.529867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:26:05.529867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:05.529867 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:05.538186 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1154/2313] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:26:05.542259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db787-64ad-474f-8f9e-d4479f5e36e1 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.557394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db787-64ad-474f-8f9e-d4479f5e36e1 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:05.562152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:05.563027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.563247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.563649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.564787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:05.567248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:05.600501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1160/2314] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:26:05 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:05.604231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.614738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafca7a-0a5f-44b4-b286-5834fdffb586 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:05.671754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:05.672542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.672829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.673101 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.673797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:05.675768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:05.697276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1155/2315] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:26:05 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:05.701503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.713272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed50af2-a6c9-4c9d-9121-add409cb0479 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:05.717779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:05.718455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.718736 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.719012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.719740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:05.721357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:05.732572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1161/2316] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:05.736652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.746645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f45f3-e0ca-4beb-a888-9792f6f83aa7 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:05.751266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11b500-4fc2-4d9c-997c-2733ca5c381a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:05.752017 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11b500-4fc2-4d9c-997c-2733ca5c381a admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.752303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11b500-4fc2-4d9c-997c-2733ca5c381a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.752582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11b500-4fc2-4d9c-997c-2733ca5c381a admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.753312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11b500-4fc2-4d9c-997c-2733ca5c381a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:05.755404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11b500-4fc2-4d9c-997c-2733ca5c381a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:05.771966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1156/2317] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:05.776308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.787022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9bf03-aea4-47cf-9b53-56141415f891 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:05.791554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:05.792462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.792747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.793022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] PATH_INFO: `/v3/projects/1d73ce3d68ac4f6e99f9a3b194cde320/users/fcabfe4f1a6b41049ca2bd37d31759aa/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.808884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d73ce3d68ac4f6e99f9a3b194cde320, user_id=fcabfe4f1a6b41049ca2bd37d31759aa, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:05.812220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:05.816440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:05.829923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1162/2318] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:26:05 2022] PUT /identity/v3/projects/1d73ce3d68ac4f6e99f9a3b194cde320/users/fcabfe4f1a6b41049ca2bd37d31759aa/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:05.835631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae485e90-3e61-4288-b530-f94c1fcac473 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:05.836024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae485e90-3e61-4288-b530-f94c1fcac473 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:05.836395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae485e90-3e61-4288-b530-f94c1fcac473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:05.854957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-ae485e90-3e61-4288-b530-f94c1fcac473 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:05.856137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ae485e90-3e61-4288-b530-f94c1fcac473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:05.866888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:05.924177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1157/2319] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:05.940291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:05.942814 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:06.007661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ab57f-b0fe-4b5c-b21b-dfb20f471f3b admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:06.011516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:06.012195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:06.012900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:06.012900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:06.022208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:06.023656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:06.033749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1163/2320] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:10.768280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa610a-8d37-4ebb-ba71-1b02692be0ca None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:10.768854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa610a-8d37-4ebb-ba71-1b02692be0ca None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:10.768854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa610a-8d37-4ebb-ba71-1b02692be0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:10.798025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-69fa610a-8d37-4ebb-ba71-1b02692be0ca None None] MFA Rules not processed for user `fcabfe4f1a6b41049ca2bd37d31759aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:10.799507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-69fa610a-8d37-4ebb-ba71-1b02692be0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:10.868864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1158/2321] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:10.873129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:10.885928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692369e-e2d8-4c52-acbe-11b6f74ab4b6 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:10.906478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:10.907191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:10.907477 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:10.907758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:10.908504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:10.912573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:10.970968 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1164/2322] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:26:10 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:10.976032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa610a-8d37-4ebb-ba71-1b02692be0ca None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:11.001217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa610a-8d37-4ebb-ba71-1b02692be0ca None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:11.077491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:11.078622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:11.079082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:11.079452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:11.080473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:11.082644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:11.115287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1159/2323] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:26:10 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:11.119258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:11.131556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c15950-14a1-430d-b880-c0d22c778880 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:11.136528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:11.137127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:11.137414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:11.137696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:11.138440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:11.140336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:11.163538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1165/2324] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:11.168501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:11.179118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31b442-c934-40ec-b6e1-67a484790c84 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:11.184166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:11.185065 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:11.185350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:11.185627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] PATH_INFO: `/v3/projects/826d699dc70e4cf1a21d13dd9a4b61e3/users/3d7b267cc708479080fb6e6c804a3dad/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:11.201355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d699dc70e4cf1a21d13dd9a4b61e3, user_id=3d7b267cc708479080fb6e6c804a3dad, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:11.204568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:11.219374 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1160/2325] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:26:11 2022] PUT /identity/v3/projects/826d699dc70e4cf1a21d13dd9a4b61e3/users/3d7b267cc708479080fb6e6c804a3dad/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:11.223234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:11.233180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18876362-c37b-42ba-8f84-7e7be9e21d73 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:11.297155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:11.297931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:11.298289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:11.298629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:11.299435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:11.301992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:11.314016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1166/2326] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:11.318519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:11.331524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534060-09ad-44af-8de4-baf05fd29c6c admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:11.336478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31447d-b970-415e-9fd1-0128b9b61e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:11.337370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31447d-b970-415e-9fd1-0128b9b61e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:11.337657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31447d-b970-415e-9fd1-0128b9b61e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:11.337939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31447d-b970-415e-9fd1-0128b9b61e5b admin admin] PATH_INFO: `/v3/projects/826d699dc70e4cf1a21d13dd9a4b61e3/users/3d7b267cc708479080fb6e6c804a3dad/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:11.340006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31447d-b970-415e-9fd1-0128b9b61e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d699dc70e4cf1a21d13dd9a4b61e3, user_id=3d7b267cc708479080fb6e6c804a3dad, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:11.342704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31447d-b970-415e-9fd1-0128b9b61e5b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:11.349897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:11.355866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1161/2327] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:26:11 2022] PUT /identity/v3/projects/826d699dc70e4cf1a21d13dd9a4b61e3/users/3d7b267cc708479080fb6e6c804a3dad/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:15.924148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:15.927601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:15.975062 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee47b9c-42d9-404a-a8b2-0bd55b3bb132 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:16.211526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:16.212564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:16.212780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:16.213061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] PATH_INFO: `/v3/users/9ea31ef824b3417abc5653a55c46e930` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:16.215982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea31ef824b3417abc5653a55c46e930)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:16.221392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:16.225371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:16.355242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea31ef824b3417abc5653a55c46e930'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:16.356242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea31ef824b3417abc5653a55c46e930'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:16.377668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea31ef824b3417abc5653a55c46e930'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:16.378641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea31ef824b3417abc5653a55c46e930'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:16.400341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-405e7e78-7ed0-4372-91b0-36ba4b47cfd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea31ef824b3417abc5653a55c46e930'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:16.415644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1167/2328] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:26:15 2022] DELETE /identity/v3/users/9ea31ef824b3417abc5653a55c46e930 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:16.419991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:16.420477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:16.420581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:16.469531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:16.470806 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:16.604472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:16.604858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:16.605264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:16.638907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1162/2329] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:16.647446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:16.649898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:16.679273 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:16.683014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:16.792677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1168/2330] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:26:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:16.800244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eaefc7-347f-4f2d-99ba-9d384bfe3b20 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:16.805582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:16.806537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:16.806904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:16.807326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:16.843556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:16.847587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:16.885367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1163/2331] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:16.900737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:16.902454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:17.045091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab7896-5f1f-4e96-ac7c-0eb2d1712445 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:17.060349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a9912-ab92-4bd1-b44f-bb5851e885d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:17.061643 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a9912-ab92-4bd1-b44f-bb5851e885d1 service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:17.062134 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a9912-ab92-4bd1-b44f-bb5851e885d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:17.062615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a9912-ab92-4bd1-b44f-bb5851e885d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:17.081200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a9912-ab92-4bd1-b44f-bb5851e885d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:17.083192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a9912-ab92-4bd1-b44f-bb5851e885d1 service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:17.099449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1169/2332] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:26:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:26:17.518273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:17.520925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:17.555668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdef9a7-1c83-4d6d-819f-64039a571d57 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:17.577277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:17.578534 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:17.579934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:17.580439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] PATH_INFO: `/v3/projects/2c0d374cce0245ebb63be064536eaaf5` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:17.583015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0d374cce0245ebb63be064536eaaf5)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:17.586983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:17.701679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0d374cce0245ebb63be064536eaaf5', 'request_id': 'req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:17.726230 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0d374cce0245ebb63be064536eaaf5', 'request_id': 'req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:17.739378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0d374cce0245ebb63be064536eaaf5', 'request_id': 'req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:17.773077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] The token cache is being invalidate because project 2c0d374cce0245ebb63be064536eaaf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:26:17.773436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:17.775364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-601fda6c-e3c5-47ee-8028-e5653e2f5bc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:17.780685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1164/2333] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:26:17 2022] DELETE /identity/v3/projects/2c0d374cce0245ebb63be064536eaaf5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:17.785501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71666a21-4177-47df-ac75-c0941d9558a3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:17.785794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71666a21-4177-47df-ac75-c0941d9558a3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:17.786079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71666a21-4177-47df-ac75-c0941d9558a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:17.802222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:17.872413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-71666a21-4177-47df-ac75-c0941d9558a3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:17.873887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-71666a21-4177-47df-ac75-c0941d9558a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:18.084621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1170/2334] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:18.086487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5027-8a62-4a2c-aa87-9904b2e15eb4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.087662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5027-8a62-4a2c-aa87-9904b2e15eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.088009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5027-8a62-4a2c-aa87-9904b2e15eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.149888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4aaf5027-8a62-4a2c-aa87-9904b2e15eb4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:18.152038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4aaf5027-8a62-4a2c-aa87-9904b2e15eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:18.174584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1165/2335] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:18.176300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71666a21-4177-47df-ac75-c0941d9558a3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.195077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71666a21-4177-47df-ac75-c0941d9558a3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.212663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.213350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.214177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.214626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.215541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:26:18.219657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.221357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.264961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1171/2336] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:26:18.265934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5027-8a62-4a2c-aa87-9904b2e15eb4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.284586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5027-8a62-4a2c-aa87-9904b2e15eb4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.290186 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.291038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.291325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.291609 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.292639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.294628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.320111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:18.349675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1166/2337] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:26:18 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:18.350976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.361925 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac8bcc-6e57-4c8a-9f7b-82fd4a2238cc admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.456196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.456912 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.457336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.457619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.458322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.460470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.501763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1172/2338] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:26:18 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:18.502982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.518617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e68e-39da-4fe9-a376-2436b947f487 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.527841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.528557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.528851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.529132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.529881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.531704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.547542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1167/2339] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:18.548258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.567806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cdc23-2944-4f0f-984f-4a0b7cb55ce3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.581524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54008698-aba0-4f37-80e7-6d36ce26ce5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.582265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54008698-aba0-4f37-80e7-6d36ce26ce5f admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.582555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54008698-aba0-4f37-80e7-6d36ce26ce5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.582835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54008698-aba0-4f37-80e7-6d36ce26ce5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.583737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54008698-aba0-4f37-80e7-6d36ce26ce5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.585398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54008698-aba0-4f37-80e7-6d36ce26ce5f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.612214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1173/2340] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:18.617367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.627720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9a7a8-9fae-47fd-bf86-a05dc9eb9d09 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.634717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.635626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.635940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.636221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] PATH_INFO: `/v3/projects/5acc90de5f4a4d6db5ed87d7f89d42ba/users/879e819a33c449039fb2e04164efbfdc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.652212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acc90de5f4a4d6db5ed87d7f89d42ba, user_id=879e819a33c449039fb2e04164efbfdc, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.654920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.683901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1168/2341] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:26:18 2022] PUT /identity/v3/projects/5acc90de5f4a4d6db5ed87d7f89d42ba/users/879e819a33c449039fb2e04164efbfdc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:18.695538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030013-17ee-402d-9ef2-38b30ae981b4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.695924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030013-17ee-402d-9ef2-38b30ae981b4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.696304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030013-17ee-402d-9ef2-38b30ae981b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.745632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-31030013-17ee-402d-9ef2-38b30ae981b4 None None] MFA Rules not processed for user `879e819a33c449039fb2e04164efbfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:18.747013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-31030013-17ee-402d-9ef2-38b30ae981b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:18.862114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:18.877554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1174/2342] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:18.901443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.903247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:19.003494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c235afc-ca47-4216-b048-1c7b0e5b98ee admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:19.021234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7be3a546aa644281a0d3594395bbdf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:19.021234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:19.021234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:19.021234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:19.043194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:19.046319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:19.060923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1169/2343] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:19.546765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea97d-d0d7-46f3-9e1a-44701a0a40fb None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:19.547253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea97d-d0d7-46f3-9e1a-44701a0a40fb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:19.547331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea97d-d0d7-46f3-9e1a-44701a0a40fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:19.581309 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-30fea97d-d0d7-46f3-9e1a-44701a0a40fb None None] MFA Rules not processed for user `3d7b267cc708479080fb6e6c804a3dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:19.583069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-30fea97d-d0d7-46f3-9e1a-44701a0a40fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:19.735161 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1175/2344] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:19.748974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:19.756515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:19.929255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562717e8-7332-492b-9ddf-3dfe04a52132 service glance] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:19.936960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8d5ce-95fd-4b05-a4bf-47567a833883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:19.937690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d5ce-95fd-4b05-a4bf-47567a833883 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:19.937963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d5ce-95fd-4b05-a4bf-47567a833883 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:19.938290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d5ce-95fd-4b05-a4bf-47567a833883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:19.945351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cfb8d5ce-95fd-4b05-a4bf-47567a833883 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:20.037461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:20.091013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8d5ce-95fd-4b05-a4bf-47567a833883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:20.093045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8d5ce-95fd-4b05-a4bf-47567a833883 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:20.118717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1170/2345] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:22.084733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801817-a9f0-434d-9a29-13fea240a94a None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:22.085518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801817-a9f0-434d-9a29-13fea240a94a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:22.085807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801817-a9f0-434d-9a29-13fea240a94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:22.132038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-dc801817-a9f0-434d-9a29-13fea240a94a None None] MFA Rules not processed for user `fcabfe4f1a6b41049ca2bd37d31759aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:22.135706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-dc801817-a9f0-434d-9a29-13fea240a94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:22.185612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1176/2346] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:24.617414 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96426b-fe11-48fa-be15-271a0086395a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.618270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96426b-fe11-48fa-be15-271a0086395a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.618625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96426b-fe11-48fa-be15-271a0086395a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.638232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-5a96426b-fe11-48fa-be15-271a0086395a None None] MFA Rules not processed for user `e8fddb1cf74d4135b41000f8a6293020`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:24.639844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5a96426b-fe11-48fa-be15-271a0086395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.721509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1171/2347] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:32.117642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=40 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:34.711707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=37 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:41.462292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdbbd1-6dc8-4513-9f50-b3562db99c32 None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.462908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdbbd1-6dc8-4513-9f50-b3562db99c32 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.462908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdbbd1-6dc8-4513-9f50-b3562db99c32 None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.464272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1177/2348] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:41.470957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1634bb-d5a4-45d4-bd73-7c83510039ab None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.472067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1634bb-d5a4-45d4-bd73-7c83510039ab None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.472658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1634bb-d5a4-45d4-bd73-7c83510039ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.514541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-df1634bb-d5a4-45d4-bd73-7c83510039ab None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:41.515760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-df1634bb-d5a4-45d4-bd73-7c83510039ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:41.525366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1172/2349] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:26:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:41.536853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdbbd1-6dc8-4513-9f50-b3562db99c32 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:41.547709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdbbd1-6dc8-4513-9f50-b3562db99c32 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:41.551778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8b9c3-c962-4be4-9834-2f3abc0e43dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7be3a546aa644281a0d3594395bbdf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:41.552447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8b9c3-c962-4be4-9834-2f3abc0e43dd service glance] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.552735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8b9c3-c962-4be4-9834-2f3abc0e43dd service glance] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.553012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8b9c3-c962-4be4-9834-2f3abc0e43dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.563323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8b9c3-c962-4be4-9834-2f3abc0e43dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:41.565587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8b9c3-c962-4be4-9834-2f3abc0e43dd service glance] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:41.577053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1178/2350] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:41.645664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587aba0-19fd-4dd1-80e9-241a64c97dc0 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.646113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587aba0-19fd-4dd1-80e9-241a64c97dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.646499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587aba0-19fd-4dd1-80e9-241a64c97dc0 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.648079 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1173/2351] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:41.653069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d2a38-0d69-463f-a73b-6248d0466382 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.653355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d2a38-0d69-463f-a73b-6248d0466382 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.653629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d2a38-0d69-463f-a73b-6248d0466382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.672245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ed4d2a38-0d69-463f-a73b-6248d0466382 None None] MFA Rules not processed for user `a364b6179abb4ad0883898090acef827`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:41.673438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ed4d2a38-0d69-463f-a73b-6248d0466382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:41.738200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1179/2352] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:26:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:41.761993 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587aba0-19fd-4dd1-80e9-241a64c97dc0 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:41.765382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8587aba0-19fd-4dd1-80e9-241a64c97dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:41.793192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587aba0-19fd-4dd1-80e9-241a64c97dc0 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:41.797502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413793b-7fc3-484c-8d82-a50a8ab708c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:41.798541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413793b-7fc3-484c-8d82-a50a8ab708c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.798850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413793b-7fc3-484c-8d82-a50a8ab708c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.799664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413793b-7fc3-484c-8d82-a50a8ab708c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.809285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413793b-7fc3-484c-8d82-a50a8ab708c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:41.811537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413793b-7fc3-484c-8d82-a50a8ab708c7 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:41.827883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1174/2353] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:41.934377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e368927-cf1f-4bbe-99d8-b301846446aa None None] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.934672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e368927-cf1f-4bbe-99d8-b301846446aa None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.935031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e368927-cf1f-4bbe-99d8-b301846446aa None None] PATH_INFO: `/` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.936463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1180/2354] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:26:41.941813 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b2b95-43ff-4e1a-8923-b5127aabdd9d None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.942103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b2b95-43ff-4e1a-8923-b5127aabdd9d None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.942383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b2b95-43ff-4e1a-8923-b5127aabdd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.960984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1b2b2b95-43ff-4e1a-8923-b5127aabdd9d None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:41.962180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1b2b2b95-43ff-4e1a-8923-b5127aabdd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:41.972008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1175/2355] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:26:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:41.982294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e368927-cf1f-4bbe-99d8-b301846446aa None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:41.994648 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e368927-cf1f-4bbe-99d8-b301846446aa None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:41.998128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae823a3-a7fc-461c-916b-4ca71d222fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:41.998770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae823a3-a7fc-461c-916b-4ca71d222fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.999053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae823a3-a7fc-461c-916b-4ca71d222fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.999621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae823a3-a7fc-461c-916b-4ca71d222fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:42.010071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae823a3-a7fc-461c-916b-4ca71d222fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:42.011488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae823a3-a7fc-461c-916b-4ca71d222fdf service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:42.022825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1181/2356] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:51.956130 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=18 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:52.017880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=35 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:58.186906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b2b95-43ff-4e1a-8923-b5127aabdd9d None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:58.347010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b2b95-43ff-4e1a-8923-b5127aabdd9d None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:58.353131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:58.354864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:58.355259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:58.355643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] PATH_INFO: `/v3/users/879e819a33c449039fb2e04164efbfdc` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:58.358713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=879e819a33c449039fb2e04164efbfdc)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:58.363234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:58.504636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '879e819a33c449039fb2e04164efbfdc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:58.510411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '879e819a33c449039fb2e04164efbfdc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:58.510411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '879e819a33c449039fb2e04164efbfdc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:58.529126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '879e819a33c449039fb2e04164efbfdc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:58.539001 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '879e819a33c449039fb2e04164efbfdc'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:58.580314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb7639-9572-476f-a92c-e995160aaa7a None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:58.581408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb7639-9572-476f-a92c-e995160aaa7a None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:58.581525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb7639-9572-476f-a92c-e995160aaa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:58.582869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1176/2357] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:26:58 2022] DELETE /identity/v3/users/879e819a33c449039fb2e04164efbfdc => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:58.621248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-bbeb7639-9572-476f-a92c-e995160aaa7a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:58.623119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bbeb7639-9572-476f-a92c-e995160aaa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:58.704239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1182/2358] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:58.712277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:58.713801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:58.804737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d1f5e-9afe-4b95-ad97-44ba6f52d015 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:58.810469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:58.811260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:58.811650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:58.812056 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:58.832965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:58.834615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:58.852960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1177/2359] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:58.869431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:58.869927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:58.870394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:58.956838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:58.958318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:59.047991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1183/2360] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:26:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:59.062566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.064268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:59.101059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:59.204632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5c79-ff5e-47fd-a1cb-72d3c36f1261 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.211542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.213210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:59.217339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39204f-0fdf-43fb-8840-da43dcfd36ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.218097 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39204f-0fdf-43fb-8840-da43dcfd36ea service swift] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.218482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39204f-0fdf-43fb-8840-da43dcfd36ea service swift] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.218877 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39204f-0fdf-43fb-8840-da43dcfd36ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.231405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd329e-01b2-4be6-86a4-d0efab55f9bd None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.231892 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39204f-0fdf-43fb-8840-da43dcfd36ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.233586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39204f-0fdf-43fb-8840-da43dcfd36ea service swift] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.236414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.237350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.237636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.237921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] PATH_INFO: `/v3/projects/5acc90de5f4a4d6db5ed87d7f89d42ba` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.239151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5acc90de5f4a4d6db5ed87d7f89d42ba)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.241183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.246010 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1178/2361] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:26:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:26:59.370470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5acc90de5f4a4d6db5ed87d7f89d42ba', 'request_id': 'req-c7b80cba-fd9c-4344-9484-216e71ebb28f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:59.378002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5acc90de5f4a4d6db5ed87d7f89d42ba', 'request_id': 'req-c7b80cba-fd9c-4344-9484-216e71ebb28f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:59.401832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5acc90de5f4a4d6db5ed87d7f89d42ba', 'request_id': 'req-c7b80cba-fd9c-4344-9484-216e71ebb28f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:59.410824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:59.430568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] The token cache is being invalidate because project 5acc90de5f4a4d6db5ed87d7f89d42ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:26:59.431061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:59.432016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c7b80cba-fd9c-4344-9484-216e71ebb28f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:59.438810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1184/2362] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:26:59 2022] DELETE /identity/v3/projects/5acc90de5f4a4d6db5ed87d7f89d42ba => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:26:59.441310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996f35-5da8-4074-a698-2a638b50e251 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.441776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996f35-5da8-4074-a698-2a638b50e251 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.442176 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996f35-5da8-4074-a698-2a638b50e251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.471729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-74996f35-5da8-4074-a698-2a638b50e251 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:59.473655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-74996f35-5da8-4074-a698-2a638b50e251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:59.608179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1179/2363] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:59.619224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e084a1c-9b6d-4fda-8ab3-8c4a43dd3773 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.619731 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e084a1c-9b6d-4fda-8ab3-8c4a43dd3773 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.619991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e084a1c-9b6d-4fda-8ab3-8c4a43dd3773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.645201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-6e084a1c-9b6d-4fda-8ab3-8c4a43dd3773 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:59.646292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6e084a1c-9b6d-4fda-8ab3-8c4a43dd3773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:59.654981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1185/2364] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:59.661440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996f35-5da8-4074-a698-2a638b50e251 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.671522 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996f35-5da8-4074-a698-2a638b50e251 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.676255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.676915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.677196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.677472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.678235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:26:59.678542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.680170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.692863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1180/2365] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:26:59.697186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e084a1c-9b6d-4fda-8ab3-8c4a43dd3773 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.708402 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e084a1c-9b6d-4fda-8ab3-8c4a43dd3773 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.713428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.714113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.714396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.714729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.715382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.717302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.746835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1186/2366] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:26:59 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:26:59.751289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.761629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454ae3a-3864-493e-b8ad-1b75f5077768 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.819129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.819850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.820153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.820427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.821127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.823121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.847051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1181/2367] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:26:59 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:26:59.848264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.858199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bfdb8-46dd-4dd9-8273-4c2c60829f80 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.862800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.863454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.864233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.864233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.864759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.866385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.876803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1187/2368] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:59.881265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.891364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420a804-90bb-4118-90f6-a2d02ca75c41 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.896709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6038d-7bc4-41d3-b4d8-7e78b8b567d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.897461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6038d-7bc4-41d3-b4d8-7e78b8b567d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.897749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6038d-7bc4-41d3-b4d8-7e78b8b567d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.898061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6038d-7bc4-41d3-b4d8-7e78b8b567d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.898800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6038d-7bc4-41d3-b4d8-7e78b8b567d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.900444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6038d-7bc4-41d3-b4d8-7e78b8b567d0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.905007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:59.921431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1182/2369] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:59.921651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:59.930179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a1f0-c4e7-42cd-9641-509bd5855713 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:59.934686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:59.935569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.935887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.936169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] PATH_INFO: `/v3/projects/0ff52e32117c43efa0eabcc3b4c75877/users/bebb779e6235425481dee8c0bbb69b1f/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:59.951640 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff52e32117c43efa0eabcc3b4c75877, user_id=bebb779e6235425481dee8c0bbb69b1f, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:59.954582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:59.967720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1188/2370] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:26:59 2022] PUT /identity/v3/projects/0ff52e32117c43efa0eabcc3b4c75877/users/bebb779e6235425481dee8c0bbb69b1f/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:59.978778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8d80-0fee-41cd-abdc-07c6cef337e9 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:59.979072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8d80-0fee-41cd-abdc-07c6cef337e9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:59.979348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8d80-0fee-41cd-abdc-07c6cef337e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:00.005976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e25c8d80-0fee-41cd-abdc-07c6cef337e9 None None] MFA Rules not processed for user `bebb779e6235425481dee8c0bbb69b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:00.007201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e25c8d80-0fee-41cd-abdc-07c6cef337e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:00.068871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1183/2371] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:26:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:00.080579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:00.082243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:00.164717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414810-b6ed-4a05-b16d-49c035231ac1 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:00.168604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:00.169269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:00.169554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:00.169835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:00.179252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:00.180718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:00.189389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1189/2372] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:27:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:10.065995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=33 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:10.185377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:10.329329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870415-67a0-4ec6-9799-397ea2176b7a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:10.329622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870415-67a0-4ec6-9799-397ea2176b7a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:10.329908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870415-67a0-4ec6-9799-397ea2176b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:10.357252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-31870415-67a0-4ec6-9799-397ea2176b7a None None] MFA Rules not processed for user `8b9bfcb74bc14e92b96e4169ed323399`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:10.358636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-31870415-67a0-4ec6-9799-397ea2176b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:10.443175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1184/2373] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:27:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:10.998289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:11.000034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:11.110352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec901b-1dc3-419b-86c2-dc7e1931b353 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:11.116960 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d64212-bf24-4a83-a3c3-3ec9f40a348f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:11.117632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d64212-bf24-4a83-a3c3-3ec9f40a348f service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:11.117926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d64212-bf24-4a83-a3c3-3ec9f40a348f service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:11.118212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d64212-bf24-4a83-a3c3-3ec9f40a348f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:11.120137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a9d64212-bf24-4a83-a3c3-3ec9f40a348f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:11.135047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d64212-bf24-4a83-a3c3-3ec9f40a348f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:11.136687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d64212-bf24-4a83-a3c3-3ec9f40a348f service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:11.147972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1190/2374] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:27:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:11.512430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31870415-67a0-4ec6-9799-397ea2176b7a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:11.574686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31870415-67a0-4ec6-9799-397ea2176b7a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:11.579859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:11.581332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:11.581635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:11.581925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] PATH_INFO: `/v3/users/bebb779e6235425481dee8c0bbb69b1f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:11.583161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=bebb779e6235425481dee8c0bbb69b1f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:11.585129 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:11.633347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bebb779e6235425481dee8c0bbb69b1f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:11.634009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bebb779e6235425481dee8c0bbb69b1f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:11.634610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bebb779e6235425481dee8c0bbb69b1f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:11.646745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bebb779e6235425481dee8c0bbb69b1f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:11.653552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bebb779e6235425481dee8c0bbb69b1f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:11.668197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1185/2375] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:27:11 2022] DELETE /identity/v3/users/bebb779e6235425481dee8c0bbb69b1f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:27:11.672568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:11.673049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:11.673144 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:11.690576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:11.691778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:11.751046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1191/2376] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:11.761021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:11.762415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:11.790419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:11.823051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a278017-38d4-4637-9bf9-22819b2567ef admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:11.827726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026906a-a0b2-4648-a943-992510cefbae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:11.828345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026906a-a0b2-4648-a943-992510cefbae service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:11.828627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026906a-a0b2-4648-a943-992510cefbae service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:11.828894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026906a-a0b2-4648-a943-992510cefbae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:11.838690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026906a-a0b2-4648-a943-992510cefbae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:11.840147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026906a-a0b2-4648-a943-992510cefbae service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:11.850454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1186/2377] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:27:12.127967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:12.129594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:12.144126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2b10-8975-4f1e-963a-29ad8fe5aaea None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:12.148958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:12.150058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:12.150471 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:12.150815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] PATH_INFO: `/v3/projects/0ff52e32117c43efa0eabcc3b4c75877` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:12.152087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff52e32117c43efa0eabcc3b4c75877)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:12.153989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:12.213282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff52e32117c43efa0eabcc3b4c75877', 'request_id': 'req-a1897576-abd1-47d5-a196-47133fbf3796'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:12.219756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff52e32117c43efa0eabcc3b4c75877', 'request_id': 'req-a1897576-abd1-47d5-a196-47133fbf3796'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:12.226412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff52e32117c43efa0eabcc3b4c75877', 'request_id': 'req-a1897576-abd1-47d5-a196-47133fbf3796'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:12.231539 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:12.245421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] The token cache is being invalidate because project 0ff52e32117c43efa0eabcc3b4c75877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:27:12.247377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:12.247377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:12.248813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1192/2378] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:27:12 2022] DELETE /identity/v3/projects/0ff52e32117c43efa0eabcc3b4c75877 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:27:21.845783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=15 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:22.249781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:25.023066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:25.023550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:25.023644 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:25.041473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:25.042662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:25.098669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1187/2379] 10.222.0.62 () {58 vars in 967 bytes} [Fri May 27 03:27:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:27:25.120877 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:25.122314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:25.189934 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1897576-abd1-47d5-a196-47133fbf3796 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:25.195487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:25.196169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:25.196458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:25.196737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:25.205139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:25.206716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:25.214549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1193/2380] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:27:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:32.905449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:32.906936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:32.963433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560124f-77ef-4640-a7a3-6e441a621f1f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:32.967120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:32.967766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:32.968206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:32.968490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:32.970341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:33.031886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:33.034072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:33.043455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1188/2381] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:27:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:35.211509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=32 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:43.039207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=75 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:15.766006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:15.769224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:15.835259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6b7b6-61a7-48a0-9cc6-403afd7d9ac0 service swift] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:15.839773 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:15.840682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.840970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.841256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] PATH_INFO: `/v3/users/8b9bfcb74bc14e92b96e4169ed323399` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.842460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9bfcb74bc14e92b96e4169ed323399)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:15.844457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:15.890818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9bfcb74bc14e92b96e4169ed323399'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:15.891467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9bfcb74bc14e92b96e4169ed323399'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:15.900105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9bfcb74bc14e92b96e4169ed323399'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:15.901006 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9bfcb74bc14e92b96e4169ed323399'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:15.913626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-01aa17b3-0d3b-429c-b1f1-2a6fa28e43b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9bfcb74bc14e92b96e4169ed323399'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:15.928574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1194/2382] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:28:15 2022] DELETE /identity/v3/users/8b9bfcb74bc14e92b96e4169ed323399 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:28:16.096512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.107709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fa018-0cec-4320-a0e3-418fba811d8b service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.169645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.170515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.170801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.171078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] PATH_INFO: `/v3/projects/aee9b49e32b848e2a75facdeef1aa8a5` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.172328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee9b49e32b848e2a75facdeef1aa8a5)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.174328 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.222793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee9b49e32b848e2a75facdeef1aa8a5', 'request_id': 'req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:16.230081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee9b49e32b848e2a75facdeef1aa8a5', 'request_id': 'req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:16.235900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee9b49e32b848e2a75facdeef1aa8a5', 'request_id': 'req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:16.246959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] The token cache is being invalidate because project aee9b49e32b848e2a75facdeef1aa8a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:28:16.247257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:16.247924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-48e5efe4-12f6-4aba-9227-7d3f9ded0c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:16.249508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1189/2383] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:28:16 2022] DELETE /identity/v3/projects/aee9b49e32b848e2a75facdeef1aa8a5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:16.254510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d028ef-f2c0-441f-93c8-c1322e630202 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.254799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d028ef-f2c0-441f-93c8-c1322e630202 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.255097 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d028ef-f2c0-441f-93c8-c1322e630202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.274062 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-31d028ef-f2c0-441f-93c8-c1322e630202 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:16.275266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-31d028ef-f2c0-441f-93c8-c1322e630202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:16.354428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1195/2384] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:16.362868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d36d7-3df4-4b5c-b77a-b0ab7ae9723b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.363169 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d36d7-3df4-4b5c-b77a-b0ab7ae9723b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.363513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d36d7-3df4-4b5c-b77a-b0ab7ae9723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.386524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-1f9d36d7-3df4-4b5c-b77a-b0ab7ae9723b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:16.388324 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1f9d36d7-3df4-4b5c-b77a-b0ab7ae9723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:16.400540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1190/2385] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:16.401678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d028ef-f2c0-441f-93c8-c1322e630202 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.412485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d028ef-f2c0-441f-93c8-c1322e630202 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.417235 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.417902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.418192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.418478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.419224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:16.419543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.421247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.433444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1196/2386] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.437798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d36d7-3df4-4b5c-b77a-b0ab7ae9723b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.448147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d36d7-3df4-4b5c-b77a-b0ab7ae9723b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.452916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.453658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.454012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.454364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.455903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.457734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.484098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1191/2387] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:28:16 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:16.488187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.498458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca94e3-74d3-4e22-915c-180a7832e523 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.502931 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.573294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.573963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.574247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.574524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.575214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.577150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.598084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1197/2388] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:28:16 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:16.602123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.612625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e37903-9d79-4d99-8475-6dedf0653c9d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.617399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.618092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.618386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.618669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.619508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.621168 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.632899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1192/2389] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:16.637027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.648695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3e314-5595-4b2d-8172-698ed3c1abd9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.653817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627bfcf-cbca-4b6b-b41a-b15bc2957bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.654566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627bfcf-cbca-4b6b-b41a-b15bc2957bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.654926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627bfcf-cbca-4b6b-b41a-b15bc2957bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.655280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627bfcf-cbca-4b6b-b41a-b15bc2957bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.656058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627bfcf-cbca-4b6b-b41a-b15bc2957bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.657669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627bfcf-cbca-4b6b-b41a-b15bc2957bb4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.671529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1198/2390] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:16.673205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.683357 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197ac5-1f04-41af-93e9-b44546330d07 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.688075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.688950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.689753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.690028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] PATH_INFO: `/v3/projects/ffc47aa302dc4c4e94c4bc7227728c40/users/5c8ad338eb7b47c18dcb86f287e28858/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.704415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc47aa302dc4c4e94c4bc7227728c40, user_id=5c8ad338eb7b47c18dcb86f287e28858, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.707602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.719795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1193/2391] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:28:16 2022] PUT /identity/v3/projects/ffc47aa302dc4c4e94c4bc7227728c40/users/5c8ad338eb7b47c18dcb86f287e28858/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:16.724808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94aecd-4681-4f12-a83d-f598ad467623 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.725131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94aecd-4681-4f12-a83d-f598ad467623 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.725415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94aecd-4681-4f12-a83d-f598ad467623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.743739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ba94aecd-4681-4f12-a83d-f598ad467623 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:16.744928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ba94aecd-4681-4f12-a83d-f598ad467623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:16.802536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1199/2392] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:28:16.812350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.815040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:16.845652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.874109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14382a02-c461-4c83-b1bd-143962699aa8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.878318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.878950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.879253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.879532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.888306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.889727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.898793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1194/2393] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:21.826423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f895-8524-412d-95e9-b88a1b8533e6 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.826957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f895-8524-412d-95e9-b88a1b8533e6 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.827069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f895-8524-412d-95e9-b88a1b8533e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.891466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d0d0f895-8524-412d-95e9-b88a1b8533e6 None None] MFA Rules not processed for user `5c8ad338eb7b47c18dcb86f287e28858`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:21.892809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d0d0f895-8524-412d-95e9-b88a1b8533e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:22.006950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.018314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1200/2394] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:22.019282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfb747-c41b-4a62-af64-a35c0b2a8174 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.025134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.025807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.026090 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.026366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.027164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.030221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.080111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1195/2395] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:28:22 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:28:22.080907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0f895-8524-412d-95e9-b88a1b8533e6 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.105857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0f895-8524-412d-95e9-b88a1b8533e6 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.122396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:22.255055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.256155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.256671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.257820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.258987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.261742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.302049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1201/2396] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:28:22 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:22.307258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.321431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c7806-414a-490c-b119-615af1ed8b0f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.342260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.342974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.343263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.343544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.344471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.346868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.377563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1196/2397] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:22.382192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.397245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c03e81-5764-45df-9e07-b08fad4fc4c4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.406318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.407229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.407658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.408162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] PATH_INFO: `/v3/projects/73b3a58043fe4fdf92acdca722e152f5/users/5558f4ffc4c947978b34643f09b20d32/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.437116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b3a58043fe4fdf92acdca722e152f5, user_id=5558f4ffc4c947978b34643f09b20d32, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.441046 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.462236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1202/2398] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:28:22 2022] PUT /identity/v3/projects/73b3a58043fe4fdf92acdca722e152f5/users/5558f4ffc4c947978b34643f09b20d32/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:22.466502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.480110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d32b1-8306-4d16-86ed-722bf613dd07 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.559947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b55c-502d-4842-ba1c-cbb902d8fa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.560812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b55c-502d-4842-ba1c-cbb902d8fa30 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.561214 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b55c-502d-4842-ba1c-cbb902d8fa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.561592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b55c-502d-4842-ba1c-cbb902d8fa30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.562439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b55c-502d-4842-ba1c-cbb902d8fa30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.564449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b55c-502d-4842-ba1c-cbb902d8fa30 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.575734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1197/2399] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:22.579663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.591403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90419a07-1db8-4403-becf-26e18e16cc89 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.598417 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468913a5-d3f1-4db7-8ffc-b25dac89805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.599321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468913a5-d3f1-4db7-8ffc-b25dac89805d admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.599603 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468913a5-d3f1-4db7-8ffc-b25dac89805d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.599987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468913a5-d3f1-4db7-8ffc-b25dac89805d admin admin] PATH_INFO: `/v3/projects/73b3a58043fe4fdf92acdca722e152f5/users/5558f4ffc4c947978b34643f09b20d32/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.601947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468913a5-d3f1-4db7-8ffc-b25dac89805d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b3a58043fe4fdf92acdca722e152f5, user_id=5558f4ffc4c947978b34643f09b20d32, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.604621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468913a5-d3f1-4db7-8ffc-b25dac89805d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.616519 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1203/2400] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:28:22 2022] PUT /identity/v3/projects/73b3a58043fe4fdf92acdca722e152f5/users/5558f4ffc4c947978b34643f09b20d32/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:24.987897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2581d8-5c4d-4875-81f5-b81ece975e51 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.988542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2581d8-5c4d-4875-81f5-b81ece975e51 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.988542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2581d8-5c4d-4875-81f5-b81ece975e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.008297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fd2581d8-5c4d-4875-81f5-b81ece975e51 None None] MFA Rules not processed for user `720bd38555404aafa93db5ae49b13acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:25.009898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fd2581d8-5c4d-4875-81f5-b81ece975e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:25.072611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1198/2401] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:27.245647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:27.246183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:27.246183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:27.298725 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] MFA Rules not processed for user `e8fddb1cf74d4135b41000f8a6293020`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:27.299996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:27.444177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1204/2402] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:28.788796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34af3dd-ef09-4481-aca9-2cec1b32022f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.789293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34af3dd-ef09-4481-aca9-2cec1b32022f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.789417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34af3dd-ef09-4481-aca9-2cec1b32022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.832369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f34af3dd-ef09-4481-aca9-2cec1b32022f None None] MFA Rules not processed for user `5558f4ffc4c947978b34643f09b20d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:28.834804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f34af3dd-ef09-4481-aca9-2cec1b32022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:28.869082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:28.920268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1199/2403] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:28.934132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:28.935780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:29.027339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4d87-5a18-4f06-b19f-54c8e4b56480 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:29.034847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:29.035861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.036000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.036412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.047947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:29.105066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:29.107137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:29.116855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1205/2404] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:28:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:38.916795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=17 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:39.112621 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=23 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:45.556855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ee51e-c3cb-4d2d-96cc-4278439d0566 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.557325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ee51e-c3cb-4d2d-96cc-4278439d0566 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.557445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ee51e-c3cb-4d2d-96cc-4278439d0566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.575627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8e6ee51e-c3cb-4d2d-96cc-4278439d0566 None None] MFA Rules not processed for user `5c8ad338eb7b47c18dcb86f287e28858`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:45.576950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8e6ee51e-c3cb-4d2d-96cc-4278439d0566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:45.583748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1200/2405] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:52.071726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.073291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:52.134121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6743766-e884-4d67-a5fd-7cd1039a9511 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.138607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb64829-c55e-4484-b29c-08cd4ef82415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.139602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb64829-c55e-4484-b29c-08cd4ef82415 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.139806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb64829-c55e-4484-b29c-08cd4ef82415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.140164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb64829-c55e-4484-b29c-08cd4ef82415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.142325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2cb64829-c55e-4484-b29c-08cd4ef82415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:52.159248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb64829-c55e-4484-b29c-08cd4ef82415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:52.160716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb64829-c55e-4484-b29c-08cd4ef82415 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:52.171673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1206/2406] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:52.205592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ee51e-c3cb-4d2d-96cc-4278439d0566 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.215404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ee51e-c3cb-4d2d-96cc-4278439d0566 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.218978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.220064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.220064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.220256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.222156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:52.236766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:52.238207 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:52.249195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1201/2407] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:59.169644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77e23-5012-4d7c-9911-f6d0dbc96196 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.170170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77e23-5012-4d7c-9911-f6d0dbc96196 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.170269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77e23-5012-4d7c-9911-f6d0dbc96196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.187721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-24a77e23-5012-4d7c-9911-f6d0dbc96196 None None] MFA Rules not processed for user `b34132c9382a44cd97bc2c4e40c97eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:59.188954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-24a77e23-5012-4d7c-9911-f6d0dbc96196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:59.245252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1207/2408] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:28:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:28:59.847503 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:59.849036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:59.924640 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9034576-f8a0-44c1-9b34-902d793dd317 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.929568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:59.930215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.930504 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.930782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.935702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:59.948751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:59.950337 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:59.959963 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1202/2409] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:28:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:01.094100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a77e23-5012-4d7c-9911-f6d0dbc96196 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:01.103712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a77e23-5012-4d7c-9911-f6d0dbc96196 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:01.107955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:01.108501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.109334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.109617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.119094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:01.120582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:01.131176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1208/2410] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:09.283385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:09.287735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=42 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:09.293553 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c1ab2-d1e4-4f63-87ed-11febc7c5a5f service placement] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:09.297622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:09.298253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.298533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.298809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.300652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:09.309906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.311507 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.320722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1203/2411] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:11.126768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=57 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:13.154152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.155611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:13.171048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397974a-7b88-4945-a4e0-a599bc40c7a0 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.177306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.177949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.178236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.178523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.180321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:13.193009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:13.194862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:13.205267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1209/2412] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:14.446389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.447887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:14.504037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2e355-a784-4ef8-8879-27c1fa3f651a service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.507559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.508229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.508514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.508790 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.510574 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:14.521038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.522459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.531200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1204/2413] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:23.200863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=6 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:24.527229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=31 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:24.646779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.657951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add3664-c955-4787-82a2-8d632618e401 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.663898 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.664536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.664823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.665104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.666861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:24.677625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.679211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.688038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1210/2414] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:34.683909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=6 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.734204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.743807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afcd33-7ce2-4000-8a71-d2153292a885 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.747329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.747992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.748275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.748549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.750300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:34.810802 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.812370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.831687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1205/2415] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:38.167212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.168700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:38.184631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd910eda-1960-4948-8fa4-15cddb8be03b service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.190483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.191135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.191418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.191696 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.193857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:38.208549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.210229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:38.222598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1211/2416] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:38.239372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.240878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:38.253543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5dbe7-245e-460e-8b71-59f1c2e1f169 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.258457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.259346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.259634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.259945 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] PATH_INFO: `/v3/users/e8fddb1cf74d4135b41000f8a6293020` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.261286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fddb1cf74d4135b41000f8a6293020)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.263232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:38.308950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fddb1cf74d4135b41000f8a6293020'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.309601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fddb1cf74d4135b41000f8a6293020'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.310211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fddb1cf74d4135b41000f8a6293020'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.322436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fddb1cf74d4135b41000f8a6293020'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.329218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c7aa4abb-7abc-459b-bf74-bad6dce08279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fddb1cf74d4135b41000f8a6293020'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.349259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1206/2417] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:29:38 2022] DELETE /identity/v3/users/e8fddb1cf74d4135b41000f8a6293020 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:38.521409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.533050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a4473-41b9-46a0-b5c0-a7604597c698 service placement] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.598100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.598970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.599257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.599538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] PATH_INFO: `/v3/projects/5e539cb7b0fe47d6bdece2805d4ad0c1` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.600799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e539cb7b0fe47d6bdece2805d4ad0c1)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.602783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:38.647715 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e539cb7b0fe47d6bdece2805d4ad0c1', 'request_id': 'req-942a62bd-ce5c-4534-887f-bffeabfcf4af'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.653707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e539cb7b0fe47d6bdece2805d4ad0c1', 'request_id': 'req-942a62bd-ce5c-4534-887f-bffeabfcf4af'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.659727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e539cb7b0fe47d6bdece2805d4ad0c1', 'request_id': 'req-942a62bd-ce5c-4534-887f-bffeabfcf4af'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.671584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] The token cache is being invalidate because project 5e539cb7b0fe47d6bdece2805d4ad0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:38.671900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.672541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-942a62bd-ce5c-4534-887f-bffeabfcf4af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:38.674111 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1212/2418] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:29:38 2022] DELETE /identity/v3/projects/5e539cb7b0fe47d6bdece2805d4ad0c1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:38.679004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3069d-2dd2-4154-93b9-fa3a311aa553 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.679293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3069d-2dd2-4154-93b9-fa3a311aa553 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.679573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3069d-2dd2-4154-93b9-fa3a311aa553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.696955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4bb3069d-2dd2-4154-93b9-fa3a311aa553 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:38.698128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4bb3069d-2dd2-4154-93b9-fa3a311aa553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:38.756266 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1207/2419] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:38.763813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704eed51-e5ff-4fec-a269-8eeeadc9636e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.764136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704eed51-e5ff-4fec-a269-8eeeadc9636e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.764442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704eed51-e5ff-4fec-a269-8eeeadc9636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.782888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-704eed51-e5ff-4fec-a269-8eeeadc9636e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:38.784110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-704eed51-e5ff-4fec-a269-8eeeadc9636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:38.791926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1213/2420] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:38.796081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3069d-2dd2-4154-93b9-fa3a311aa553 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.805445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3069d-2dd2-4154-93b9-fa3a311aa553 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.810027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.810659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.810939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.811213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.811976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:38.812284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.814397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:38.815988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:38.830205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1208/2421] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:38.834377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704eed51-e5ff-4fec-a269-8eeeadc9636e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.844075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704eed51-e5ff-4fec-a269-8eeeadc9636e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.848707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.849396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.849677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.849951 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.850645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.852632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:38.880707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1214/2422] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:29:38 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:38.884866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.895306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f532f-83db-402d-9484-50d01bfe7cc9 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.952396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.953057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.953339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.953614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.954455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.956899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:38.976825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1209/2423] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:29:38 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:38.981210 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.991192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf06e-9ebd-4b05-b04e-6b50f46b57a8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.995746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.996423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.996708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.996985 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.997713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.999306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.010349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1215/2424] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:39.013839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.024250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdcd74-0b32-49ad-bad1-4f46d7b3b454 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.028857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64851048-df58-4586-bb25-1d28b6e2396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.029505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851048-df58-4586-bb25-1d28b6e2396b admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.029791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851048-df58-4586-bb25-1d28b6e2396b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.030067 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851048-df58-4586-bb25-1d28b6e2396b admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.030819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64851048-df58-4586-bb25-1d28b6e2396b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.032439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64851048-df58-4586-bb25-1d28b6e2396b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.042034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1210/2425] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:39.045886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.055966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411cbbe-438c-4a94-a196-d2839852bfe8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.060604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.061472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.061754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.062056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] PATH_INFO: `/v3/projects/e9893ffa59dc401cb2f6b953edfd6c15/users/2999115bc4fc4a8fbac238d94b58f4d6/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.077900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9893ffa59dc401cb2f6b953edfd6c15, user_id=2999115bc4fc4a8fbac238d94b58f4d6, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.081179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.093520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1216/2426] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:39 2022] PUT /identity/v3/projects/e9893ffa59dc401cb2f6b953edfd6c15/users/2999115bc4fc4a8fbac238d94b58f4d6/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:39.103758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f545c-6b47-4704-97c8-edbcd6b9b84e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.104083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f545c-6b47-4704-97c8-edbcd6b9b84e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.104367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f545c-6b47-4704-97c8-edbcd6b9b84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.130879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2a9f545c-6b47-4704-97c8-edbcd6b9b84e None None] MFA Rules not processed for user `2999115bc4fc4a8fbac238d94b58f4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:39.132106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2a9f545c-6b47-4704-97c8-edbcd6b9b84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:39.200678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1211/2427] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:39.211647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.213362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:39.227833 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:39.289828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38f11-cd7a-4cf2-be6e-068da9f18bfc admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.295382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173c17e-60d5-4ac5-8bbb-1bf671707313 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.296058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c17e-60d5-4ac5-8bbb-1bf671707313 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.296343 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c17e-60d5-4ac5-8bbb-1bf671707313 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.296619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c17e-60d5-4ac5-8bbb-1bf671707313 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.305054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173c17e-60d5-4ac5-8bbb-1bf671707313 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.306649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173c17e-60d5-4ac5-8bbb-1bf671707313 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.315774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1217/2428] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:39.508832 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f545c-6b47-4704-97c8-edbcd6b9b84e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.570250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f545c-6b47-4704-97c8-edbcd6b9b84e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.574770 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.575638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.575967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.576251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] PATH_INFO: `/v3/users/2999115bc4fc4a8fbac238d94b58f4d6` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.577567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2999115bc4fc4a8fbac238d94b58f4d6)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.579672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.582018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:39.626153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2999115bc4fc4a8fbac238d94b58f4d6'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:39.626820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2999115bc4fc4a8fbac238d94b58f4d6'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:39.627442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2999115bc4fc4a8fbac238d94b58f4d6'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:39.639246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2999115bc4fc4a8fbac238d94b58f4d6'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:39.646016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2999115bc4fc4a8fbac238d94b58f4d6'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:39.658581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1212/2429] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:29:39 2022] DELETE /identity/v3/users/2999115bc4fc4a8fbac238d94b58f4d6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:39.663188 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.663481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.663763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.680821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:39.682022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:39.740217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1218/2430] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:39.749604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.751000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:39.807111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9bc6-20e2-4793-a66b-6845ec9ac36b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.810798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bb65e-89d2-4860-ad92-61e58f1a1956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.811439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb65e-89d2-4860-ad92-61e58f1a1956 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.811728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb65e-89d2-4860-ad92-61e58f1a1956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.812053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb65e-89d2-4860-ad92-61e58f1a1956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.821680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bb65e-89d2-4860-ad92-61e58f1a1956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.823126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bb65e-89d2-4860-ad92-61e58f1a1956 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.832939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1213/2431] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:40.105844 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.107367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:40.120915 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e56a1b-427c-47a5-8c65-dea92400efb4 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.125574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.126500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.126784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.127080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] PATH_INFO: `/v3/projects/e9893ffa59dc401cb2f6b953edfd6c15` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.128296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9893ffa59dc401cb2f6b953edfd6c15)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.130197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.179329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9893ffa59dc401cb2f6b953edfd6c15', 'request_id': 'req-0de627c5-1d23-4b25-ab64-2b86d5154c37'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:40.185421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9893ffa59dc401cb2f6b953edfd6c15', 'request_id': 'req-0de627c5-1d23-4b25-ab64-2b86d5154c37'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:40.191410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9893ffa59dc401cb2f6b953edfd6c15', 'request_id': 'req-0de627c5-1d23-4b25-ab64-2b86d5154c37'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:40.203447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] The token cache is being invalidate because project e9893ffa59dc401cb2f6b953edfd6c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:40.203734 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:40.204387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0de627c5-1d23-4b25-ab64-2b86d5154c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:40.206553 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1219/2432] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:29:40 2022] DELETE /identity/v3/projects/e9893ffa59dc401cb2f6b953edfd6c15 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:40.211536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0be3ee-304a-4dfc-8c21-778d40becdbd None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.211849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0be3ee-304a-4dfc-8c21-778d40becdbd None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.212145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0be3ee-304a-4dfc-8c21-778d40becdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.229237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8b0be3ee-304a-4dfc-8c21-778d40becdbd None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:40.230420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8b0be3ee-304a-4dfc-8c21-778d40becdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:40.287805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1214/2433] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:40.296535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596b0a3-02f6-4a48-85c4-8f070c077c98 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.297028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596b0a3-02f6-4a48-85c4-8f070c077c98 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.297294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596b0a3-02f6-4a48-85c4-8f070c077c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.317428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0596b0a3-02f6-4a48-85c4-8f070c077c98 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:40.318642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0596b0a3-02f6-4a48-85c4-8f070c077c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:40.326853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1220/2434] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:40.331497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0be3ee-304a-4dfc-8c21-778d40becdbd None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.341053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0be3ee-304a-4dfc-8c21-778d40becdbd None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.345703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.346397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.346681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.346959 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.347719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:40.348058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.349646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.361451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1215/2435] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:40.365692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596b0a3-02f6-4a48-85c4-8f070c077c98 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.374943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596b0a3-02f6-4a48-85c4-8f070c077c98 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.380231 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.380893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.381180 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.381463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.382173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.384110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.398343 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.411726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1221/2436] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:40.415791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.426027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6ee4e-367a-4d28-968d-9cdc71b90fd2 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.433769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.486221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.486930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.487223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.487500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.488234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.490148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.510423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1216/2437] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:40.514572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.525644 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907fc10-e6a1-4c15-a88c-78825877a85f admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.530200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.530847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.531130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.531408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.532167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.533747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.544108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1222/2438] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:29:40.547988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.558699 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f39b6-c65c-4e52-82e7-de2bcb112a66 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.563319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20a37-c63c-41cf-8b7b-c03f2267a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.564021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20a37-c63c-41cf-8b7b-c03f2267a390 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.564307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20a37-c63c-41cf-8b7b-c03f2267a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.564581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20a37-c63c-41cf-8b7b-c03f2267a390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.565862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20a37-c63c-41cf-8b7b-c03f2267a390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.567720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20a37-c63c-41cf-8b7b-c03f2267a390 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.577636 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1217/2439] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:40.581843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.591759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593942f-a311-44cb-b3a8-a208d2e5cc41 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.596365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.597241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.597527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.597809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] PATH_INFO: `/v3/projects/03d3e29948ae48759f9a291257f3704c/users/5c9af880bdd34936b02fa43ac65581b7/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.612558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d3e29948ae48759f9a291257f3704c, user_id=5c9af880bdd34936b02fa43ac65581b7, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.615195 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.628747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1223/2440] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:40 2022] PUT /identity/v3/projects/03d3e29948ae48759f9a291257f3704c/users/5c9af880bdd34936b02fa43ac65581b7/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:40.638956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621f25-c5cd-4912-b88f-06df193d7a6a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.639250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621f25-c5cd-4912-b88f-06df193d7a6a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.639533 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621f25-c5cd-4912-b88f-06df193d7a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.665838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-54621f25-c5cd-4912-b88f-06df193d7a6a None None] MFA Rules not processed for user `5c9af880bdd34936b02fa43ac65581b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:40.667018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-54621f25-c5cd-4912-b88f-06df193d7a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:40.730108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1218/2441] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:40.734523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.795250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f315bb2-bdeb-4092-a261-9ffe4ad3c985 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.800341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.800995 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.801286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.801574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.802269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.804203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.829892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1224/2442] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:40.833368 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54621f25-c5cd-4912-b88f-06df193d7a6a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.844052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54621f25-c5cd-4912-b88f-06df193d7a6a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.899811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.900492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.900775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.901053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.901748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.903657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.923588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1219/2443] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:40.928375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.939373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6299f9a-fc89-41c0-a533-dc35e29a117d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.944947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.945802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.946080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.946387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.947112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.948749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.959312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1225/2444] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:40.963445 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.973692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9922-4371-40ae-b8f3-a16428636e75 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.978203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.979068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.979353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.979628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] PATH_INFO: `/v3/projects/7954df1241034e35b91566aa98cb34f0/users/7db0a0ad7cdb47c2b8058bd146602ecc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.982629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.996007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7954df1241034e35b91566aa98cb34f0, user_id=7db0a0ad7cdb47c2b8058bd146602ecc, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.998667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.011030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1220/2445] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:40 2022] PUT /identity/v3/projects/7954df1241034e35b91566aa98cb34f0/users/7db0a0ad7cdb47c2b8058bd146602ecc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:41.015004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.024772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebee83-9942-49e8-83e1-6d50227cd542 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.079589 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7154529-2c83-4623-95cf-e8ec627d582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.080225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7154529-2c83-4623-95cf-e8ec627d582f admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.080507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7154529-2c83-4623-95cf-e8ec627d582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.080782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7154529-2c83-4623-95cf-e8ec627d582f admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.081503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7154529-2c83-4623-95cf-e8ec627d582f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.083088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7154529-2c83-4623-95cf-e8ec627d582f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.092494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1226/2446] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:41.097236 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.107291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdebb51-d351-4a27-a95b-86d72f171b73 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.112465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.113338 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.113622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.113900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] PATH_INFO: `/v3/projects/7954df1241034e35b91566aa98cb34f0/users/7db0a0ad7cdb47c2b8058bd146602ecc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.115893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7954df1241034e35b91566aa98cb34f0, user_id=7db0a0ad7cdb47c2b8058bd146602ecc, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.118548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.128837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1221/2447] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:41 2022] PUT /identity/v3/projects/7954df1241034e35b91566aa98cb34f0/users/7db0a0ad7cdb47c2b8058bd146602ecc/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:41.138788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921f63-9292-41cc-aa33-38d3841611cc None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.139078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921f63-9292-41cc-aa33-38d3841611cc None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.139357 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921f63-9292-41cc-aa33-38d3841611cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.166016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-01921f63-9292-41cc-aa33-38d3841611cc None None] MFA Rules not processed for user `7db0a0ad7cdb47c2b8058bd146602ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:41.167199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-01921f63-9292-41cc-aa33-38d3841611cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:41.183690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:41.234770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1227/2448] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:29:41.248044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.249749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:41.309760 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a22251-c73a-4b72-bb54-bac864f67916 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.313361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3c7c-05c1-413a-a0e4-01215b567c11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.313997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3c7c-05c1-413a-a0e4-01215b567c11 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.314284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3c7c-05c1-413a-a0e4-01215b567c11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.314578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3c7c-05c1-413a-a0e4-01215b567c11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.323832 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3c7c-05c1-413a-a0e4-01215b567c11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.325361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3c7c-05c1-413a-a0e4-01215b567c11 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.335070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1222/2449] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:41.404764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01921f63-9292-41cc-aa33-38d3841611cc None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.414763 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01921f63-9292-41cc-aa33-38d3841611cc None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.418331 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.418957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.419243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.419517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.477129 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.478567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.487928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1228/2450] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:48.352655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.353551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.354057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.375475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:48.376748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:48.442299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1223/2451] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:29:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:48.501538 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.503515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:48.605168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7fabf-1153-4860-9cc5-ff309e5151eb service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.610400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.611059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.611344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.611626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.626509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.628325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.639444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1229/2452] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:29:49.070933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:49.072420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:49.177255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76ef6-a60b-4530-a117-c72391a6e993 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.185450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894534c-a076-425d-9e9a-089c44070539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.186215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894534c-a076-425d-9e9a-089c44070539 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.186572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894534c-a076-425d-9e9a-089c44070539 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.186924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894534c-a076-425d-9e9a-089c44070539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.188859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2894534c-a076-425d-9e9a-089c44070539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:49.202893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894534c-a076-425d-9e9a-089c44070539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.206723 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894534c-a076-425d-9e9a-089c44070539 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:49.217579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1224/2453] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:51.480119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.532887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.553095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbb4e9-97d1-4f67-b16f-48b95cd0563a service swift] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.558806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.559998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.560510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.560792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] PATH_INFO: `/v3/users/5c9af880bdd34936b02fa43ac65581b7` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.562009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9af880bdd34936b02fa43ac65581b7)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.564093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.631496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9af880bdd34936b02fa43ac65581b7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:51.632447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9af880bdd34936b02fa43ac65581b7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:51.642151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9af880bdd34936b02fa43ac65581b7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:51.642765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9af880bdd34936b02fa43ac65581b7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:51.661426 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9af880bdd34936b02fa43ac65581b7'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:51.677826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1230/2454] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:29:51 2022] DELETE /identity/v3/users/5c9af880bdd34936b02fa43ac65581b7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:51.682471 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.683599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.683599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.690044 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.704451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:51.706125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:51.773857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1225/2455] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:51.783328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.785381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:51.854069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24fc1e-84cb-4094-bd76-2c386660d8c4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.857742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.858400 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.858688 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.859033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.869894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.871382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.882700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1231/2456] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:52.358787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.359769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:52.379045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e92ab5-f00f-4001-aee4-4df4e539f9d7 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.399994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.401610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.402086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.402576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] PATH_INFO: `/v3/projects/03d3e29948ae48759f9a291257f3704c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.404280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d3e29948ae48759f9a291257f3704c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.408041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.462042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d3e29948ae48759f9a291257f3704c', 'request_id': 'req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.473281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d3e29948ae48759f9a291257f3704c', 'request_id': 'req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.481350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d3e29948ae48759f9a291257f3704c', 'request_id': 'req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.497323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] The token cache is being invalidate because project 03d3e29948ae48759f9a291257f3704c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:52.497703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.498427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.500080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1226/2457] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:29:52 2022] DELETE /identity/v3/projects/03d3e29948ae48759f9a291257f3704c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:52.504221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.515063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce2663-5a20-4d3f-9f71-b77f89c52c64 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.585875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.586998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.587286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.587563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] PATH_INFO: `/v3/users/7db0a0ad7cdb47c2b8058bd146602ecc` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.588813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db0a0ad7cdb47c2b8058bd146602ecc)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.590756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.597582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:52.653831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db0a0ad7cdb47c2b8058bd146602ecc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.654802 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db0a0ad7cdb47c2b8058bd146602ecc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.668588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db0a0ad7cdb47c2b8058bd146602ecc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.668588 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db0a0ad7cdb47c2b8058bd146602ecc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.685017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b7f97e62-7770-476b-9418-3d6c0d4b4889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db0a0ad7cdb47c2b8058bd146602ecc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:52.705094 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1232/2458] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:29:52 2022] DELETE /identity/v3/users/7db0a0ad7cdb47c2b8058bd146602ecc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:53.036415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.037985 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:53.052948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b0dd-10b5-4c79-8dfd-c401efa8ea23 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.136317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.137196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.137484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.137765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] PATH_INFO: `/v3/projects/7954df1241034e35b91566aa98cb34f0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.138977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7954df1241034e35b91566aa98cb34f0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.141009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.180776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:53.222746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7954df1241034e35b91566aa98cb34f0', 'request_id': 'req-f1caa363-0817-4b86-859b-bdc66552c147'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:53.235845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7954df1241034e35b91566aa98cb34f0', 'request_id': 'req-f1caa363-0817-4b86-859b-bdc66552c147'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:53.247861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7954df1241034e35b91566aa98cb34f0', 'request_id': 'req-f1caa363-0817-4b86-859b-bdc66552c147'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:53.261847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] The token cache is being invalidate because project 7954df1241034e35b91566aa98cb34f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:53.262340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:53.263235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f1caa363-0817-4b86-859b-bdc66552c147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:53.267902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1227/2459] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:29:53 2022] DELETE /identity/v3/projects/7954df1241034e35b91566aa98cb34f0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:53.272515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1150-e46a-4d21-8a1a-c58387dd112f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.272805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1150-e46a-4d21-8a1a-c58387dd112f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.273139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1150-e46a-4d21-8a1a-c58387dd112f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.308661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-494a1150-e46a-4d21-8a1a-c58387dd112f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:53.309361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-494a1150-e46a-4d21-8a1a-c58387dd112f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:53.413240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1233/2460] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:53.422684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab743-5614-4b23-a9a3-ebc3a4702f17 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.423371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab743-5614-4b23-a9a3-ebc3a4702f17 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.423904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab743-5614-4b23-a9a3-ebc3a4702f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.460181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-943ab743-5614-4b23-a9a3-ebc3a4702f17 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:53.461410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-943ab743-5614-4b23-a9a3-ebc3a4702f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:53.476064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1228/2461] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:53.480272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1150-e46a-4d21-8a1a-c58387dd112f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.491178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1150-e46a-4d21-8a1a-c58387dd112f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.507517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.508347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.508708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.509058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.509875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:53.510256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.517051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.535340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1234/2462] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:53.540940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab743-5614-4b23-a9a3-ebc3a4702f17 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.551115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab743-5614-4b23-a9a3-ebc3a4702f17 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.559423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.560549 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.560841 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.561123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.562006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.565177 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.611085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1229/2463] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:29:53 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:53.612427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.627028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c90e5-76cf-4ebe-9405-0e7e8089c420 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.700505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.701103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.701395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.701674 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.702380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.704376 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.724190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:53.731616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.732892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1235/2464] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:29:53 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:53.744900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d64722-7461-4fa1-9ba2-3d7b1aaf3841 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.750031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.750947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.751429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.751795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.753139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.754911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.771235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1230/2465] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:29:53.772672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.784504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6ddd-1650-4ed2-a0ac-2d17422070d7 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.789109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474a0b-0a3d-459f-a2b5-62b35bed9db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.789794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474a0b-0a3d-459f-a2b5-62b35bed9db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.790092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474a0b-0a3d-459f-a2b5-62b35bed9db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.790358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474a0b-0a3d-459f-a2b5-62b35bed9db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.791123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce474a0b-0a3d-459f-a2b5-62b35bed9db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.792767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce474a0b-0a3d-459f-a2b5-62b35bed9db9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.808853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1236/2466] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:53.812835 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.827567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39117283-1c71-4a00-aacc-dd159d64500d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.833024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.834027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.834441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.834826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] PATH_INFO: `/v3/projects/4a85496179864c0087de2620f8b94bab/users/d0df04bdd4ee4dada15f5e8d86122b32/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.853851 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a85496179864c0087de2620f8b94bab, user_id=d0df04bdd4ee4dada15f5e8d86122b32, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.856866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.878060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1231/2467] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:53 2022] PUT /identity/v3/projects/4a85496179864c0087de2620f8b94bab/users/d0df04bdd4ee4dada15f5e8d86122b32/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:53.889360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b54100-08b3-44ea-9e22-b42a29397888 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.889658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b54100-08b3-44ea-9e22-b42a29397888 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.889969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b54100-08b3-44ea-9e22-b42a29397888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.926083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-40b54100-08b3-44ea-9e22-b42a29397888 None None] MFA Rules not processed for user `d0df04bdd4ee4dada15f5e8d86122b32`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:53.927398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-40b54100-08b3-44ea-9e22-b42a29397888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:54.007857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1237/2468] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:29:54.012255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.108825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ebee3-d048-492a-b31b-d3292b14e45e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.113918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.114889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.114889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.115087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.115774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.117750 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.147717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1232/2469] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:29:54 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:54.152001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b54100-08b3-44ea-9e22-b42a29397888 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.165837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b54100-08b3-44ea-9e22-b42a29397888 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.237211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.238023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.238391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.238741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.239650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.241712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.268275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1238/2470] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:29:54 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:54.271537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.282439 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2508a9f-de4f-41c8-8ac2-dbbb480ded4c admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.288392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.289082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.289500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.289798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.290544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.292232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.310842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.311317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1233/2471] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:54.323999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bddb-dcbc-4a6f-952b-bf173c10ba69 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.332223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.333266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.333717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.334002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] PATH_INFO: `/v3/projects/be969bf915d84a52ab8dd9c1e80e3c7b/users/8e93075151ad4410a649b9431545a97c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.351695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be969bf915d84a52ab8dd9c1e80e3c7b, user_id=8e93075151ad4410a649b9431545a97c, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.354401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.370552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1239/2472] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:54 2022] PUT /identity/v3/projects/be969bf915d84a52ab8dd9c1e80e3c7b/users/8e93075151ad4410a649b9431545a97c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:54.374456 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.389099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246147e-9137-464e-8ee5-e58b90e4ab3a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.421928 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:54.464347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a875fa-36f3-4624-aaa5-23d24a7ac4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.465054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a875fa-36f3-4624-aaa5-23d24a7ac4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.465429 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a875fa-36f3-4624-aaa5-23d24a7ac4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.465720 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a875fa-36f3-4624-aaa5-23d24a7ac4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.466486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a875fa-36f3-4624-aaa5-23d24a7ac4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.469359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a875fa-36f3-4624-aaa5-23d24a7ac4c6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.483836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.484583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1234/2473] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:54.496574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b445eb0-426b-4278-adec-a9616d72b556 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.501462 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.502943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.503290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.503636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] PATH_INFO: `/v3/projects/be969bf915d84a52ab8dd9c1e80e3c7b/users/8e93075151ad4410a649b9431545a97c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.505840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be969bf915d84a52ab8dd9c1e80e3c7b, user_id=8e93075151ad4410a649b9431545a97c, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.508792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.521541 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1240/2474] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:54 2022] PUT /identity/v3/projects/be969bf915d84a52ab8dd9c1e80e3c7b/users/8e93075151ad4410a649b9431545a97c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:54.533588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc2fb-fc09-4490-8c4b-cc6a9a7a978e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.533875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc2fb-fc09-4490-8c4b-cc6a9a7a978e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.534160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc2fb-fc09-4490-8c4b-cc6a9a7a978e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.566876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-45dbc2fb-fc09-4490-8c4b-cc6a9a7a978e None None] MFA Rules not processed for user `8e93075151ad4410a649b9431545a97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:54.568383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-45dbc2fb-fc09-4490-8c4b-cc6a9a7a978e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:54.702085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1235/2475] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:54.709568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.711340 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:54.749064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:54.787263 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfd1bd-808e-4508-b472-aa5706d6be85 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.792946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f322b-dbbb-4638-938c-1a3c8af32f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.792946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f322b-dbbb-4638-938c-1a3c8af32f3a service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.792946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f322b-dbbb-4638-938c-1a3c8af32f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.792946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f322b-dbbb-4638-938c-1a3c8af32f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.808241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f322b-dbbb-4638-938c-1a3c8af32f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.811339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f322b-dbbb-4638-938c-1a3c8af32f3a service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.823635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1241/2476] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:55.347624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc2fb-fc09-4490-8c4b-cc6a9a7a978e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.421679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc2fb-fc09-4490-8c4b-cc6a9a7a978e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.427300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.428220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.428536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.428820 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] PATH_INFO: `/v3/users/d0df04bdd4ee4dada15f5e8d86122b32` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.430061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0df04bdd4ee4dada15f5e8d86122b32)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.432237 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.485977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0df04bdd4ee4dada15f5e8d86122b32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:55.487649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0df04bdd4ee4dada15f5e8d86122b32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:55.488521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0df04bdd4ee4dada15f5e8d86122b32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:55.495526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:55.506163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0df04bdd4ee4dada15f5e8d86122b32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:55.516630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0df04bdd4ee4dada15f5e8d86122b32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:55.533126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1236/2477] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:29:55 2022] DELETE /identity/v3/users/d0df04bdd4ee4dada15f5e8d86122b32 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:55.538078 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.538584 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.538691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.560431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:55.561944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:55.633261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1242/2478] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:55.643973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.645696 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:55.746401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c941b-3201-47e2-a9fd-35d8e4cdcbde admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.755464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.756455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.756933 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.757220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.777179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.779925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.793164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1237/2479] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:56.195962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.197254 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:56.223031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc88c2-5f91-4646-abfd-162585d40148 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.229466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.230401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.230752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.231095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] PATH_INFO: `/v3/projects/4a85496179864c0087de2620f8b94bab` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.232394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a85496179864c0087de2620f8b94bab)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.234494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.292182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a85496179864c0087de2620f8b94bab', 'request_id': 'req-4d76e43b-959f-4981-aa8b-060112f7d452'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.300137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a85496179864c0087de2620f8b94bab', 'request_id': 'req-4d76e43b-959f-4981-aa8b-060112f7d452'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.309123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a85496179864c0087de2620f8b94bab', 'request_id': 'req-4d76e43b-959f-4981-aa8b-060112f7d452'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.325585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] The token cache is being invalidate because project 4a85496179864c0087de2620f8b94bab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:56.325780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.326406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.328003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1243/2480] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:29:56 2022] DELETE /identity/v3/projects/4a85496179864c0087de2620f8b94bab => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:56.332633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.344075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d081b-9a90-436b-a64f-5b897316b0bb service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.411386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.412354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.412637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.412913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] PATH_INFO: `/v3/users/8e93075151ad4410a649b9431545a97c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.414300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e93075151ad4410a649b9431545a97c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.416538 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.476998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e93075151ad4410a649b9431545a97c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.478045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e93075151ad4410a649b9431545a97c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.478730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e93075151ad4410a649b9431545a97c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.495869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e93075151ad4410a649b9431545a97c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.503782 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e93075151ad4410a649b9431545a97c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:56.505520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.520786 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1238/2481] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:29:56 2022] DELETE /identity/v3/users/8e93075151ad4410a649b9431545a97c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:56.770770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.772358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:56.834745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.836526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:56.865806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.895222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e43b-959f-4981-aa8b-060112f7d452 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.996074 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186fa3d-6455-498a-ad18-9e534014c819 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.006923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.008478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.009612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.011142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] PATH_INFO: `/v3/projects/be969bf915d84a52ab8dd9c1e80e3c7b` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.013495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be969bf915d84a52ab8dd9c1e80e3c7b)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.015914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.031845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bbee5-e87a-4834-895f-d67bcc0245cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.032510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bbee5-e87a-4834-895f-d67bcc0245cc service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.032795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bbee5-e87a-4834-895f-d67bcc0245cc service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.033586 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bbee5-e87a-4834-895f-d67bcc0245cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.035667 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-007bbee5-e87a-4834-895f-d67bcc0245cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:57.114132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be969bf915d84a52ab8dd9c1e80e3c7b', 'request_id': 'req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:57.128847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be969bf915d84a52ab8dd9c1e80e3c7b', 'request_id': 'req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:57.143282 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be969bf915d84a52ab8dd9c1e80e3c7b', 'request_id': 'req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:57.144459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bbee5-e87a-4834-895f-d67bcc0245cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.146104 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bbee5-e87a-4834-895f-d67bcc0245cc service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.162732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] The token cache is being invalidate because project be969bf915d84a52ab8dd9c1e80e3c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:57.163029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:57.164812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-63edb9e0-75d4-4722-aa1e-fa9dcdaa9ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:57.167136 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1239/2482] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:29:56 2022] DELETE /identity/v3/projects/be969bf915d84a52ab8dd9c1e80e3c7b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:57.179752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1244/2483] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:57.188832 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e16a15-f504-41ff-b860-95b4fd80e7db None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.189353 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e16a15-f504-41ff-b860-95b4fd80e7db None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.189634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e16a15-f504-41ff-b860-95b4fd80e7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.230393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c9e16a15-f504-41ff-b860-95b4fd80e7db None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:57.231602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c9e16a15-f504-41ff-b860-95b4fd80e7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:57.319602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1240/2484] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:57.328039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea67fbc-a590-4578-98c0-899d3cf2ac6b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.328342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea67fbc-a590-4578-98c0-899d3cf2ac6b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.329531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea67fbc-a590-4578-98c0-899d3cf2ac6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.355174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-cea67fbc-a590-4578-98c0-899d3cf2ac6b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:57.358950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-cea67fbc-a590-4578-98c0-899d3cf2ac6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:57.379406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1245/2485] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:57.380632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e16a15-f504-41ff-b860-95b4fd80e7db None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.396971 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e16a15-f504-41ff-b860-95b4fd80e7db None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.411248 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.413847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.414413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.414685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.431485 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:57.432071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.435030 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.457013 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1241/2486] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:57.461395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea67fbc-a590-4578-98c0-899d3cf2ac6b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.474133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea67fbc-a590-4578-98c0-899d3cf2ac6b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.480344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.481336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.481795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.482278 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.483375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.486034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.526514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1246/2487] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:29:57 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:57.527040 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.537962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ec4e-2ff6-40b4-b3e8-281e8c52c41b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.545340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.645157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.645878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.646251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.646447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.647279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.650151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.673509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1242/2488] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:29:57 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:57.677596 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.691419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024799bd-b67e-4bbc-9eba-1f8b82bd4beb admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.696675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.697339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.697629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.697948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.698739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.700765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.714308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1247/2489] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:57.718559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.730085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ac36e-5f81-48ea-a7a8-114c2213021b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.735389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83821e63-bf2d-4d0e-8493-d6de6cd62aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.736099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83821e63-bf2d-4d0e-8493-d6de6cd62aba admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.737046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83821e63-bf2d-4d0e-8493-d6de6cd62aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.737219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83821e63-bf2d-4d0e-8493-d6de6cd62aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.738087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83821e63-bf2d-4d0e-8493-d6de6cd62aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.739708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83821e63-bf2d-4d0e-8493-d6de6cd62aba admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.750738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1243/2490] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:57.755051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.765412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed8c48-aff3-4617-9825-1e7cf0619ae5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.770220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.771172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.771750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.771893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] PATH_INFO: `/v3/projects/c31ee61e2cfa45efbdf3832983797329/users/e77e872b905a4fe49cc1d122a616e887/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.788445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31ee61e2cfa45efbdf3832983797329, user_id=e77e872b905a4fe49cc1d122a616e887, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.791310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.805576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1248/2491] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:57 2022] PUT /identity/v3/projects/c31ee61e2cfa45efbdf3832983797329/users/e77e872b905a4fe49cc1d122a616e887/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:57.815792 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfdeaf-f47b-4d34-b992-12f478c15b7b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.816444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfdeaf-f47b-4d34-b992-12f478c15b7b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.816731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfdeaf-f47b-4d34-b992-12f478c15b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.851749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-06cfdeaf-f47b-4d34-b992-12f478c15b7b None None] MFA Rules not processed for user `e77e872b905a4fe49cc1d122a616e887`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:57.853061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-06cfdeaf-f47b-4d34-b992-12f478c15b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:57.926235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1244/2492] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:57.933558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.964939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.998727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368169d-1ac9-43c8-9fc3-8fc9d3be8e47 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.004615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.005335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.006117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.006117 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.006671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.008698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.036012 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1249/2493] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:29:57 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:58.040218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfdeaf-f47b-4d34-b992-12f478c15b7b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.050777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfdeaf-f47b-4d34-b992-12f478c15b7b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.108182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.108844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.109125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.109396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.110092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.112121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.131509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1245/2494] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:29:58 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:58.135641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.146371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cf580-4ecb-4000-a6e5-f93beacf0a91 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.151139 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.151737 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.152126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.152411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.153172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.154781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.166990 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1250/2495] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:58.171198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.181673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f22634-4d6a-4a20-bc38-b0139627cd86 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.187482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.188452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.188741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.189025 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] PATH_INFO: `/v3/projects/d19f0ea396a2424ea5cbf086bb28fce9/users/230dc703cad049139f379227265d5d9c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.192872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.209220 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19f0ea396a2424ea5cbf086bb28fce9, user_id=230dc703cad049139f379227265d5d9c, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.212417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.235371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1246/2496] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:58 2022] PUT /identity/v3/projects/d19f0ea396a2424ea5cbf086bb28fce9/users/230dc703cad049139f379227265d5d9c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:58.237438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.249454 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08cddf-7637-4c8f-84cf-ddba1f8605ac admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.307530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfd4f2-824b-40ab-9574-118d8d6ea5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.308241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfd4f2-824b-40ab-9574-118d8d6ea5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.308529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfd4f2-824b-40ab-9574-118d8d6ea5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.309654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfd4f2-824b-40ab-9574-118d8d6ea5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.309964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfd4f2-824b-40ab-9574-118d8d6ea5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.311577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfd4f2-824b-40ab-9574-118d8d6ea5b7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.323374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1251/2497] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:58.327061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.337855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad240a-30b0-453d-b8c5-1d16ca0ee6fe admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.343078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.344191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.344716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.344716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] PATH_INFO: `/v3/projects/d19f0ea396a2424ea5cbf086bb28fce9/users/230dc703cad049139f379227265d5d9c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.347446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19f0ea396a2424ea5cbf086bb28fce9, user_id=230dc703cad049139f379227265d5d9c, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.354000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.366192 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1247/2498] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:29:58 2022] PUT /identity/v3/projects/d19f0ea396a2424ea5cbf086bb28fce9/users/230dc703cad049139f379227265d5d9c/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:58.378367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92665b-31fb-4ce0-a16f-79255c4ee430 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.378658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92665b-31fb-4ce0-a16f-79255c4ee430 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.378938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92665b-31fb-4ce0-a16f-79255c4ee430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.407868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fb92665b-31fb-4ce0-a16f-79255c4ee430 None None] MFA Rules not processed for user `230dc703cad049139f379227265d5d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:58.409088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fb92665b-31fb-4ce0-a16f-79255c4ee430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.498940 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1252/2499] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:29:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:58.511261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.513080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.575147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22153ea1-bf32-4723-b913-4dfaf188c748 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.580264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.580816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.581379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.581683 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.594081 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.595633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.605648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1248/2500] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:58.664175 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92665b-31fb-4ce0-a16f-79255c4ee430 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.686164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92665b-31fb-4ce0-a16f-79255c4ee430 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.693707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6f7c-63b2-4846-a432-6d871bfb0d5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.694563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6f7c-63b2-4846-a432-6d871bfb0d5c service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.695064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6f7c-63b2-4846-a432-6d871bfb0d5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.695450 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6f7c-63b2-4846-a432-6d871bfb0d5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.723072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.769886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab6f7c-63b2-4846-a432-6d871bfb0d5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.771334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab6f7c-63b2-4846-a432-6d871bfb0d5c service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.781825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1253/2501] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:02.246781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.368243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d338d-62bb-49aa-8414-44b69495cbef service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.374625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.375671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.376153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.376797 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] PATH_INFO: `/v3/users/e77e872b905a4fe49cc1d122a616e887` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.378554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77e872b905a4fe49cc1d122a616e887)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.380543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.439037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77e872b905a4fe49cc1d122a616e887'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:02.439737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77e872b905a4fe49cc1d122a616e887'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:02.440372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77e872b905a4fe49cc1d122a616e887'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:02.455777 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77e872b905a4fe49cc1d122a616e887'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:02.463559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77e872b905a4fe49cc1d122a616e887'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:02.465316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:02.479070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1249/2502] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:02 2022] DELETE /identity/v3/users/e77e872b905a4fe49cc1d122a616e887 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:02.483424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.483989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.484107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.501782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:02.503126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:02.565604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1254/2503] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:02.575306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.576767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:02.656114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ea4cf-09d8-42f9-96aa-ea1ed18e0bd2 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.661112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.661827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.662273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.662562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.682014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.682014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.685261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1250/2504] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:02.969936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.971722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:02.985061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee20c5-37f3-4fa5-8d22-1cbd0eebe1c3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.990167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.991069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.991354 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.991631 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] PATH_INFO: `/v3/projects/c31ee61e2cfa45efbdf3832983797329` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.992854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31ee61e2cfa45efbdf3832983797329)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.994771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.058676 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31ee61e2cfa45efbdf3832983797329', 'request_id': 'req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.066330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31ee61e2cfa45efbdf3832983797329', 'request_id': 'req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.074712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31ee61e2cfa45efbdf3832983797329', 'request_id': 'req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.098942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] The token cache is being invalidate because project c31ee61e2cfa45efbdf3832983797329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:03.099281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.099941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.102125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1255/2505] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:02 2022] DELETE /identity/v3/projects/c31ee61e2cfa45efbdf3832983797329 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:03.106107 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.117430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175625a0-7089-4443-8ea9-f4c903dcc97f service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.202120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.203709 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.204049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.204501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] PATH_INFO: `/v3/users/230dc703cad049139f379227265d5d9c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.205822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=230dc703cad049139f379227265d5d9c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.208062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.270825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230dc703cad049139f379227265d5d9c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.271962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230dc703cad049139f379227265d5d9c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.272938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230dc703cad049139f379227265d5d9c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.290992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230dc703cad049139f379227265d5d9c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.300978 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-00253fff-0a8e-4b9f-b4e3-b2a7aa43ba2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230dc703cad049139f379227265d5d9c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.321048 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1251/2506] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:03 2022] DELETE /identity/v3/users/230dc703cad049139f379227265d5d9c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:03.700332 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.701930 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:03.723756 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f042a-2b8c-4b3f-8b3d-69eb97fc3c8b admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.780321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:03.799081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.800251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.800850 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.801292 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] PATH_INFO: `/v3/projects/d19f0ea396a2424ea5cbf086bb28fce9` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.809710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19f0ea396a2424ea5cbf086bb28fce9)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.812956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.893881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19f0ea396a2424ea5cbf086bb28fce9', 'request_id': 'req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.901122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19f0ea396a2424ea5cbf086bb28fce9', 'request_id': 'req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.907378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19f0ea396a2424ea5cbf086bb28fce9', 'request_id': 'req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.922123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] The token cache is being invalidate because project d19f0ea396a2424ea5cbf086bb28fce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:03.922605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.924011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8d4a6f57-032e-4a7c-8a2f-b8a3b920cd53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:03.925635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1256/2507] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:03 2022] DELETE /identity/v3/projects/d19f0ea396a2424ea5cbf086bb28fce9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:03.930413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08f2aa-b169-4ac4-8cf9-2dac992fa72a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.930879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08f2aa-b169-4ac4-8cf9-2dac992fa72a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.931224 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08f2aa-b169-4ac4-8cf9-2dac992fa72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.950504 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8a08f2aa-b169-4ac4-8cf9-2dac992fa72a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:03.951695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8a08f2aa-b169-4ac4-8cf9-2dac992fa72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:03.960751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:04.013623 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1252/2508] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:04.022021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5cd9e-4bfe-445e-8a76-17a2f0e83be3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.022401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5cd9e-4bfe-445e-8a76-17a2f0e83be3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.022695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5cd9e-4bfe-445e-8a76-17a2f0e83be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.043082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-6af5cd9e-4bfe-445e-8a76-17a2f0e83be3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:04.044359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6af5cd9e-4bfe-445e-8a76-17a2f0e83be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:04.052573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1257/2509] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:04.056815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08f2aa-b169-4ac4-8cf9-2dac992fa72a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.066524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08f2aa-b169-4ac4-8cf9-2dac992fa72a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.071060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.071703 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.072014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.072294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.073024 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:04.073332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.074926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.086846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1253/2510] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:04.091140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5cd9e-4bfe-445e-8a76-17a2f0e83be3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.101147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5cd9e-4bfe-445e-8a76-17a2f0e83be3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.106692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.107411 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.107702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.108014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.108809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.110767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.136647 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1258/2511] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:04.140887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.152424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273e819-3d95-4f29-97bb-252c6ea3b137 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.219907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.220611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.220900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.221181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.221891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.223909 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.250690 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1254/2512] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:04.252740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.270071 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cafbe-927b-4c3e-b0d8-7c1a4535eadd admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.275707 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.276753 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.277171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.277636 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.278872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.280227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.293510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1259/2513] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:04.298062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.308868 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad185a3-244b-4ed4-b254-3ccff58e9b8e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.315318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7eab9-758c-4156-b5ed-be20db7f3b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.316396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7eab9-758c-4156-b5ed-be20db7f3b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.316774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7eab9-758c-4156-b5ed-be20db7f3b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.317123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7eab9-758c-4156-b5ed-be20db7f3b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.317929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7eab9-758c-4156-b5ed-be20db7f3b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.319748 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7eab9-758c-4156-b5ed-be20db7f3b3c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.331243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1255/2514] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:04.335542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.346558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856d69d-a098-4650-9c3b-72f19a8ee7df admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.351487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.352433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.352723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.353003 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] PATH_INFO: `/v3/projects/c1c9b14528964a61b626fd6f145c3ff8/users/88df01a67109498e86eeee67f964b7b9/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.369100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c9b14528964a61b626fd6f145c3ff8, user_id=88df01a67109498e86eeee67f964b7b9, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.371764 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.385845 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1260/2515] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:04 2022] PUT /identity/v3/projects/c1c9b14528964a61b626fd6f145c3ff8/users/88df01a67109498e86eeee67f964b7b9/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:04.402923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb871071-aa14-4ecf-917b-7c5a2791bf58 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.403214 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb871071-aa14-4ecf-917b-7c5a2791bf58 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.403496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb871071-aa14-4ecf-917b-7c5a2791bf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.429325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-bb871071-aa14-4ecf-917b-7c5a2791bf58 None None] MFA Rules not processed for user `88df01a67109498e86eeee67f964b7b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:04.430589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-bb871071-aa14-4ecf-917b-7c5a2791bf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:04.501794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1256/2516] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:04.506511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.574149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a24d44-e091-4316-adff-398ac2399fc6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.589229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.590009 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.590305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.590572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.591350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.593296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.614512 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:04.624025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1261/2517] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:04.628917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb871071-aa14-4ecf-917b-7c5a2791bf58 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.640863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb871071-aa14-4ecf-917b-7c5a2791bf58 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.646259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:04.715235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.715943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.716232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.716510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.717205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.719156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.740630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1257/2518] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:04.744626 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.756592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e744-8aad-4b0b-9622-1b5ec04cc164 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.763733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.764523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.764820 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.765106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.767015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.769106 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.784530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1262/2519] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:04.785903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.796031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215230ca-80ce-41f0-ac20-cff9095475d4 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.801232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.802115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.802399 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.802684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] PATH_INFO: `/v3/projects/ab0d94cbe91640df878da703bb9c294d/users/3462d5d192854cffb662d417ef54710b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.820021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0d94cbe91640df878da703bb9c294d, user_id=3462d5d192854cffb662d417ef54710b, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.823847 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.838021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1258/2520] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:04 2022] PUT /identity/v3/projects/ab0d94cbe91640df878da703bb9c294d/users/3462d5d192854cffb662d417ef54710b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:04.841954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.852205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38990-fb0e-48a9-be37-e0f1237c192e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.909100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e43d9b-081e-4df8-acf5-573255d6b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.909772 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e43d9b-081e-4df8-acf5-573255d6b562 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.910152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e43d9b-081e-4df8-acf5-573255d6b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.910437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e43d9b-081e-4df8-acf5-573255d6b562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.911167 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e43d9b-081e-4df8-acf5-573255d6b562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.912817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e43d9b-081e-4df8-acf5-573255d6b562 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.923858 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1263/2521] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:04.931026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.942434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8b450-d1a3-42cc-8a63-56db7b742935 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.947167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.949343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.949343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.949343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] PATH_INFO: `/v3/projects/ab0d94cbe91640df878da703bb9c294d/users/3462d5d192854cffb662d417ef54710b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.950861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0d94cbe91640df878da703bb9c294d, user_id=3462d5d192854cffb662d417ef54710b, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.953654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.963925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1259/2522] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:04 2022] PUT /identity/v3/projects/ab0d94cbe91640df878da703bb9c294d/users/3462d5d192854cffb662d417ef54710b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:04.975206 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65abc7-d2de-4fc8-8fd5-df61707b0347 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.975504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65abc7-d2de-4fc8-8fd5-df61707b0347 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.975839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65abc7-d2de-4fc8-8fd5-df61707b0347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.005146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7e65abc7-d2de-4fc8-8fd5-df61707b0347 None None] MFA Rules not processed for user `3462d5d192854cffb662d417ef54710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:05.006377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7e65abc7-d2de-4fc8-8fd5-df61707b0347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.088056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1264/2523] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:05.100879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.102701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.176194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695527d3-fad3-471d-a083-2b7c296af99d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.180212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70935a9-07f7-4c62-936b-d12536467735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.180973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70935a9-07f7-4c62-936b-d12536467735 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.181335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70935a9-07f7-4c62-936b-d12536467735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.181620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70935a9-07f7-4c62-936b-d12536467735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.193535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70935a9-07f7-4c62-936b-d12536467735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.195113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70935a9-07f7-4c62-936b-d12536467735 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.205714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1260/2524] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:05.604524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65abc7-d2de-4fc8-8fd5-df61707b0347 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.673988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65abc7-d2de-4fc8-8fd5-df61707b0347 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.682336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.683529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.683953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.685216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] PATH_INFO: `/v3/users/88df01a67109498e86eeee67f964b7b9` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.686686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88df01a67109498e86eeee67f964b7b9)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.689412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.704087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:05.710478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341fb61-5f8c-4600-a5b1-48d17dac20bc None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.710981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341fb61-5f8c-4600-a5b1-48d17dac20bc None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.711078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341fb61-5f8c-4600-a5b1-48d17dac20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.731587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-2341fb61-5f8c-4600-a5b1-48d17dac20bc None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:05.732920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2341fb61-5f8c-4600-a5b1-48d17dac20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.779391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88df01a67109498e86eeee67f964b7b9'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:05.787222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88df01a67109498e86eeee67f964b7b9'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:05.801408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88df01a67109498e86eeee67f964b7b9'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:05.802262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88df01a67109498e86eeee67f964b7b9'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:05.825021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-fd55b6f8-e5b6-47ff-ab67-5a9a8af99be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88df01a67109498e86eeee67f964b7b9'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:05.835741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1261/2525] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:30:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:05.837254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd94529-8b90-45ba-9ed4-81f5836b4582 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.837584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd94529-8b90-45ba-9ed4-81f5836b4582 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.837866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd94529-8b90-45ba-9ed4-81f5836b4582 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.839326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1262/2526] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:05.845122 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1265/2527] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:05 2022] DELETE /identity/v3/users/88df01a67109498e86eeee67f964b7b9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:05.849552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.850966 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.851565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.851712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.852079 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.852426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.873119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:05.874005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:05.874555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.906810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] MFA Rules not processed for user `a364b6179abb4ad0883898090acef827`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:05.908250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.966690 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1266/2528] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:05.972933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.975126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.986310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1263/2529] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:30:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:05.987428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.989072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:06.000379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10987ab-a4b4-4eb2-bf6b-2cf33e6009f4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.004734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.005380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.005665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.005938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.015537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.017743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.029269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1264/2530] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:06.030075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.031440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:06.043366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85991aa5-9af5-4b1b-9141-e84aea8e22d8 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.047404 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87862614-3e8b-4f06-844f-73f35411b992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.048764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862614-3e8b-4f06-844f-73f35411b992 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.049119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862614-3e8b-4f06-844f-73f35411b992 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.049466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862614-3e8b-4f06-844f-73f35411b992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.057984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87862614-3e8b-4f06-844f-73f35411b992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.059539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87862614-3e8b-4f06-844f-73f35411b992 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.067100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa37b0-1c39-4069-b448-ffc51fe384ab None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.069071 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1265/2531] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:06.075602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d26cc-21af-490c-bf03-74cb045b1725 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.076394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d26cc-21af-490c-bf03-74cb045b1725 service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.076761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d26cc-21af-490c-bf03-74cb045b1725 service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.077042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d26cc-21af-490c-bf03-74cb045b1725 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.121614 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:06.158816 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d26cc-21af-490c-bf03-74cb045b1725 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.160817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d26cc-21af-490c-bf03-74cb045b1725 service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.169944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1267/2532] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:30:06.175477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f0d75-7ea3-477d-aa0c-5e5619dd72d7 None None] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.175722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f0d75-7ea3-477d-aa0c-5e5619dd72d7 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.176027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f0d75-7ea3-477d-aa0c-5e5619dd72d7 None None] PATH_INFO: `/` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.177519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1266/2533] 10.222.0.62 () {54 vars in 921 bytes} [Fri May 27 03:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:06.185457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.185748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.186029 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.210351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] MFA Rules not processed for user `edc938bed535461e974daf01bb99e9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:06.211526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:06.324031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1268/2534] 10.222.0.62 () {58 vars in 1018 bytes} [Fri May 27 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:06.327031 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f0d75-7ea3-477d-aa0c-5e5619dd72d7 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.339372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f0d75-7ea3-477d-aa0c-5e5619dd72d7 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.349116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.350520 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.351319 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.351927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.392856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.398663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.425015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.426617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:06.448232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1267/2535] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:06.450329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99cf0-384b-47b3-96d0-a6d6c803d244 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.469529 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.470952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.471239 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.471517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] PATH_INFO: `/v3/projects/c1c9b14528964a61b626fd6f145c3ff8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.472961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c9b14528964a61b626fd6f145c3ff8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.474946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.584521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c9b14528964a61b626fd6f145c3ff8', 'request_id': 'req-cb44d094-3b07-4e34-9603-2914644c2892'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.594702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c9b14528964a61b626fd6f145c3ff8', 'request_id': 'req-cb44d094-3b07-4e34-9603-2914644c2892'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.602059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c9b14528964a61b626fd6f145c3ff8', 'request_id': 'req-cb44d094-3b07-4e34-9603-2914644c2892'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.625552 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] The token cache is being invalidate because project c1c9b14528964a61b626fd6f145c3ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:06.625735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.628096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.632100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1269/2536] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:06 2022] DELETE /identity/v3/projects/c1c9b14528964a61b626fd6f145c3ff8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:06.636624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.652369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ac112-314f-4a46-a503-5a7e79c676d5 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.789152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.790057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.790342 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.790627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] PATH_INFO: `/v3/users/3462d5d192854cffb662d417ef54710b` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.791986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3462d5d192854cffb662d417ef54710b)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.794562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.907038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3462d5d192854cffb662d417ef54710b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.908547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3462d5d192854cffb662d417ef54710b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.909258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3462d5d192854cffb662d417ef54710b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.932688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3462d5d192854cffb662d417ef54710b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.935410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:06.945389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-71929dc5-67c3-48ff-8248-6c3c675d44c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3462d5d192854cffb662d417ef54710b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:06.962290 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1268/2537] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:06 2022] DELETE /identity/v3/users/3462d5d192854cffb662d417ef54710b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:07.442600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.444201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:07.481818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44d094-3b07-4e34-9603-2914644c2892 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.571777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:07.591574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.592487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.592778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.593059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] PATH_INFO: `/v3/projects/ab0d94cbe91640df878da703bb9c294d` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.594247 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0d94cbe91640df878da703bb9c294d)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.596429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.650168 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0d94cbe91640df878da703bb9c294d', 'request_id': 'req-723574ed-ab5c-4e84-b0a5-6270e48018d6'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.656392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0d94cbe91640df878da703bb9c294d', 'request_id': 'req-723574ed-ab5c-4e84-b0a5-6270e48018d6'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.662739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0d94cbe91640df878da703bb9c294d', 'request_id': 'req-723574ed-ab5c-4e84-b0a5-6270e48018d6'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.675140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] The token cache is being invalidate because project ab0d94cbe91640df878da703bb9c294d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:07.675430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.676082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-723574ed-ab5c-4e84-b0a5-6270e48018d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.677618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1270/2538] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:07 2022] DELETE /identity/v3/projects/ab0d94cbe91640df878da703bb9c294d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:07.682635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27fe10-7fa5-4369-a3a5-c5455aaef93b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.683008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27fe10-7fa5-4369-a3a5-c5455aaef93b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.683286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27fe10-7fa5-4369-a3a5-c5455aaef93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.714519 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-0a27fe10-7fa5-4369-a3a5-c5455aaef93b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:07.716078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0a27fe10-7fa5-4369-a3a5-c5455aaef93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:07.780968 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1269/2539] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:07.790384 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1266d-c489-4471-b1fc-dd0201977653 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.790740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1266d-c489-4471-b1fc-dd0201977653 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.791025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1266d-c489-4471-b1fc-dd0201977653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.810436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-59a1266d-c489-4471-b1fc-dd0201977653 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:07.811632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-59a1266d-c489-4471-b1fc-dd0201977653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:07.819678 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1271/2540] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:07.824625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27fe10-7fa5-4369-a3a5-c5455aaef93b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.836744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27fe10-7fa5-4369-a3a5-c5455aaef93b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.842238 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.842904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.843287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.843558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.844523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:07.845383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.847209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.859153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1270/2541] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:07.863530 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1266d-c489-4471-b1fc-dd0201977653 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.873311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1266d-c489-4471-b1fc-dd0201977653 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.878280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.878943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.879253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.879499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.880248 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.882147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.911368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1272/2542] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:07 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:07.915413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.934323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825a2b7-5949-4f44-a585-94cf838dba41 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.001163 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.001759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.002050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.002331 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.003043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.005042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.027954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1271/2543] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:07 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:08.029508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.040891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388922d1-95e2-4977-b8ff-c4f9d8d93e96 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.045691 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.046370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.046657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.046943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.047906 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.050030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.061256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1273/2544] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:08.065294 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.075800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92e072-2e81-4796-94c0-8d270cceaa49 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.080537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db5951-0e7e-434c-b15d-9e0d8ae55759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.081201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db5951-0e7e-434c-b15d-9e0d8ae55759 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.081487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db5951-0e7e-434c-b15d-9e0d8ae55759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.081764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db5951-0e7e-434c-b15d-9e0d8ae55759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.082528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db5951-0e7e-434c-b15d-9e0d8ae55759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.084200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db5951-0e7e-434c-b15d-9e0d8ae55759 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.094306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1272/2545] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:08.099011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.109363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c0d4b-84e8-4d3b-9f70-197f6f2286cb admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.114775 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.115670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.116034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.116318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] PATH_INFO: `/v3/projects/630d1410a09249498c57aef3de6b8d46/users/cb116c983f654e33804e6b6cf43dc20d/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.140165 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630d1410a09249498c57aef3de6b8d46, user_id=cb116c983f654e33804e6b6cf43dc20d, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.143466 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.156682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1274/2546] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:08 2022] PUT /identity/v3/projects/630d1410a09249498c57aef3de6b8d46/users/cb116c983f654e33804e6b6cf43dc20d/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:08.168149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a22b0-06a1-4f50-9802-ee7999c773b6 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.168443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a22b0-06a1-4f50-9802-ee7999c773b6 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.168743 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a22b0-06a1-4f50-9802-ee7999c773b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.197240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-641a22b0-06a1-4f50-9802-ee7999c773b6 None None] MFA Rules not processed for user `cb116c983f654e33804e6b6cf43dc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:08.198453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-641a22b0-06a1-4f50-9802-ee7999c773b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:08.248226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.333642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1273/2547] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:08.344735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.348380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:08.476189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75119b19-d958-411c-ad23-44bdc1a055d7 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.481314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162480f-3608-4432-a8ac-2deb49062821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.482103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162480f-3608-4432-a8ac-2deb49062821 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.482458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162480f-3608-4432-a8ac-2deb49062821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.482787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162480f-3608-4432-a8ac-2deb49062821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.499958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162480f-3608-4432-a8ac-2deb49062821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.501543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162480f-3608-4432-a8ac-2deb49062821 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.511335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1275/2548] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:08.531628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a22b0-06a1-4f50-9802-ee7999c773b6 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.610602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a22b0-06a1-4f50-9802-ee7999c773b6 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.616713 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.617629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.618108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.618386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] PATH_INFO: `/v3/users/cb116c983f654e33804e6b6cf43dc20d` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.619685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb116c983f654e33804e6b6cf43dc20d)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.621653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.685259 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb116c983f654e33804e6b6cf43dc20d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.686049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb116c983f654e33804e6b6cf43dc20d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.686660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb116c983f654e33804e6b6cf43dc20d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.700229 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb116c983f654e33804e6b6cf43dc20d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.708281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb116c983f654e33804e6b6cf43dc20d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.728894 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1274/2549] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:08 2022] DELETE /identity/v3/users/cb116c983f654e33804e6b6cf43dc20d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:08.733824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.734183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.734630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.763944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:08.765491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:08.776922 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.845388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1276/2550] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:08.852569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.854200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:08.919935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b6f66-ceb3-48c3-bd75-610baffc27a8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.920896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67979d-82e8-4ea9-9417-9cf7f00004ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.922078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67979d-82e8-4ea9-9417-9cf7f00004ef service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.922493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67979d-82e8-4ea9-9417-9cf7f00004ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.922889 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67979d-82e8-4ea9-9417-9cf7f00004ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.932908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67979d-82e8-4ea9-9417-9cf7f00004ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.934355 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67979d-82e8-4ea9-9417-9cf7f00004ef service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.944895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1275/2551] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:09.295713 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.297508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:09.326024 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fba67-e1ed-444b-ae96-aef72f5be71c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.332670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.333573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.333862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.334146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] PATH_INFO: `/v3/projects/630d1410a09249498c57aef3de6b8d46` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.335412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=630d1410a09249498c57aef3de6b8d46)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.337564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.424137 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630d1410a09249498c57aef3de6b8d46', 'request_id': 'req-d82632ec-bcea-4298-a232-0032bf1ee444'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:09.430813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630d1410a09249498c57aef3de6b8d46', 'request_id': 'req-d82632ec-bcea-4298-a232-0032bf1ee444'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:09.437055 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630d1410a09249498c57aef3de6b8d46', 'request_id': 'req-d82632ec-bcea-4298-a232-0032bf1ee444'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:09.450428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] The token cache is being invalidate because project 630d1410a09249498c57aef3de6b8d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:09.450721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:09.451361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d82632ec-bcea-4298-a232-0032bf1ee444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:09.452991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1277/2552] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:09 2022] DELETE /identity/v3/projects/630d1410a09249498c57aef3de6b8d46 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:09.457919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7133b-143c-4766-8ade-1448876e9bc4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.458209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7133b-143c-4766-8ade-1448876e9bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.458488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7133b-143c-4766-8ade-1448876e9bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.464913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:09.493976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f9f7133b-143c-4766-8ade-1448876e9bc4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:09.495846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f9f7133b-143c-4766-8ade-1448876e9bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:09.576183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1276/2553] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:09.583401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa9ec-3588-44df-b8bb-676213f2cc97 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.583928 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa9ec-3588-44df-b8bb-676213f2cc97 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.584154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa9ec-3588-44df-b8bb-676213f2cc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.616853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fe9aa9ec-3588-44df-b8bb-676213f2cc97 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:09.618066 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fe9aa9ec-3588-44df-b8bb-676213f2cc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:09.638902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1278/2554] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:09.644478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7133b-143c-4766-8ade-1448876e9bc4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.661073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7133b-143c-4766-8ade-1448876e9bc4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.671341 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.672253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.672624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.673551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.674402 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:09.674812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.676759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.724828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1277/2555] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:09.726518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9aa9ec-3588-44df-b8bb-676213f2cc97 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.741950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9aa9ec-3588-44df-b8bb-676213f2cc97 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.748430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.749822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.750350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.756383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.757413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.760836 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.801635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1279/2556] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:09.802922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.815048 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902c4b9-6606-4d30-862f-b561d3fc0e68 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.880325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.881027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.881393 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.881745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.882523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.884576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.914484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1278/2557] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:09.921212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.931395 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f88ac-4d58-421f-90b4-5d820833d2ad admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.936158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.937324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.937324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.937516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.938138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.939784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.959443 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1280/2558] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:09.963689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.977432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210927a4-b522-45d2-9960-09d5188d25e1 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.983041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77920463-6c64-4723-b385-a1f857e4a746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.983795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77920463-6c64-4723-b385-a1f857e4a746 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.984180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77920463-6c64-4723-b385-a1f857e4a746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.984910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77920463-6c64-4723-b385-a1f857e4a746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.985828 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77920463-6c64-4723-b385-a1f857e4a746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.987541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77920463-6c64-4723-b385-a1f857e4a746 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.000870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1279/2559] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:10.005327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.015692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82983813-9179-443a-baf3-f5d2fa6ae6f8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.021339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.022251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.022540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.022824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] PATH_INFO: `/v3/projects/4cdf91ae2f2e4bfe939172e950d08aa0/users/7928c5a77799487bbdd661ff85bf7ed5/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.043970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdf91ae2f2e4bfe939172e950d08aa0, user_id=7928c5a77799487bbdd661ff85bf7ed5, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.047629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.063030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1281/2560] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:10 2022] PUT /identity/v3/projects/4cdf91ae2f2e4bfe939172e950d08aa0/users/7928c5a77799487bbdd661ff85bf7ed5/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:10.073693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225c50-d970-44cf-9794-caf15a03dcf8 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.073987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225c50-d970-44cf-9794-caf15a03dcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.074271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225c50-d970-44cf-9794-caf15a03dcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.107737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b5225c50-d970-44cf-9794-caf15a03dcf8 None None] MFA Rules not processed for user `7928c5a77799487bbdd661ff85bf7ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:10.109070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b5225c50-d970-44cf-9794-caf15a03dcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:10.173680 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:10.179448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1280/2561] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:10.192428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.194271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:10.234373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:10.286320 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d892d1-0ef1-4e32-9b6a-39c9969b731a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.289980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7be3a546aa644281a0d3594395bbdf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.290624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.290910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.291194 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.299846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.301289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.310253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1282/2562] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:20.183275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:20.306022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=20 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:23.531296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0615-2bc5-4a42-8492-d3d54b58f350 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:23.531766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0615-2bc5-4a42-8492-d3d54b58f350 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:23.531896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0615-2bc5-4a42-8492-d3d54b58f350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:23.552579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-545c0615-2bc5-4a42-8492-d3d54b58f350 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:23.552735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-545c0615-2bc5-4a42-8492-d3d54b58f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:23.617783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1281/2563] 10.222.0.62 () {58 vars in 967 bytes} [Fri May 27 03:30:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:23.648459 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:23.649972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:23.735577 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb79bb-e4ee-4f16-960c-8f441cd66788 service glance] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:23.742746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d293612-2a45-4704-af25-17b22b1f9f73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:23.743803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d293612-2a45-4704-af25-17b22b1f9f73 service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:23.744264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d293612-2a45-4704-af25-17b22b1f9f73 service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:23.744746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d293612-2a45-4704-af25-17b22b1f9f73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:23.754754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d293612-2a45-4704-af25-17b22b1f9f73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:23.756388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d293612-2a45-4704-af25-17b22b1f9f73 service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:23.767398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1283/2564] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:30:27.050048 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c0615-2bc5-4a42-8492-d3d54b58f350 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.112275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c0615-2bc5-4a42-8492-d3d54b58f350 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.116834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.117730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.117995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.118270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] PATH_INFO: `/v3/users/7928c5a77799487bbdd661ff85bf7ed5` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.119468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] RBAC: Authorizing `identity:delete_user(user_id=7928c5a77799487bbdd661ff85bf7ed5)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.121708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.177555 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7928c5a77799487bbdd661ff85bf7ed5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.185268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7928c5a77799487bbdd661ff85bf7ed5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.186491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7928c5a77799487bbdd661ff85bf7ed5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.200089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7928c5a77799487bbdd661ff85bf7ed5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.207626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7928c5a77799487bbdd661ff85bf7ed5'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.223765 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1282/2565] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:27 2022] DELETE /identity/v3/users/7928c5a77799487bbdd661ff85bf7ed5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:27.228308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.228702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.229205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.249933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:27.251151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.312228 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1284/2566] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:27.322869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.324332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.351911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.384279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ca06b-9be1-4344-a3f8-126200cae5db admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.387938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3a5eb-6651-46ce-8d3f-d5eeee300b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.388587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3a5eb-6651-46ce-8d3f-d5eeee300b21 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.388872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3a5eb-6651-46ce-8d3f-d5eeee300b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.389149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3a5eb-6651-46ce-8d3f-d5eeee300b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.398488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3a5eb-6651-46ce-8d3f-d5eeee300b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.399962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3a5eb-6651-46ce-8d3f-d5eeee300b21 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.410003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1283/2567] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:27.843063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.845176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.857184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76923ea-8e99-4577-b669-ba72a5348efe None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.862914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.863996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.864860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.865149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] PATH_INFO: `/v3/projects/4cdf91ae2f2e4bfe939172e950d08aa0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.866578 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdf91ae2f2e4bfe939172e950d08aa0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.868563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.929835 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdf91ae2f2e4bfe939172e950d08aa0', 'request_id': 'req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.936564 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdf91ae2f2e4bfe939172e950d08aa0', 'request_id': 'req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.943630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdf91ae2f2e4bfe939172e950d08aa0', 'request_id': 'req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.961172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] The token cache is being invalidate because project 4cdf91ae2f2e4bfe939172e950d08aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:27.961598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.962352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0a055fb3-2fcd-4f22-a7fb-a09825e0c0b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.964510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1285/2568] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:27 2022] DELETE /identity/v3/projects/4cdf91ae2f2e4bfe939172e950d08aa0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:27.971371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafff4c-019e-466e-868b-d42c9c261721 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.971873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafff4c-019e-466e-868b-d42c9c261721 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.972258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafff4c-019e-466e-868b-d42c9c261721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.993155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-eeafff4c-019e-466e-868b-d42c9c261721 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:27.994559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-eeafff4c-019e-466e-868b-d42c9c261721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.068069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1284/2569] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:28.129666 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2119-48b2-4d0c-9dd7-b1073e8cdee0 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.129956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2119-48b2-4d0c-9dd7-b1073e8cdee0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.130526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2119-48b2-4d0c-9dd7-b1073e8cdee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.150592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-6e2a2119-48b2-4d0c-9dd7-b1073e8cdee0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.151810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6e2a2119-48b2-4d0c-9dd7-b1073e8cdee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.159811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1286/2570] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:28.164174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafff4c-019e-466e-868b-d42c9c261721 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.174118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafff4c-019e-466e-868b-d42c9c261721 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.178785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.179454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.179741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.180061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.180801 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:28.181122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.182733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.195536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1285/2571] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:28.199809 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2119-48b2-4d0c-9dd7-b1073e8cdee0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.209868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2119-48b2-4d0c-9dd7-b1073e8cdee0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.214520 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.215232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.215444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.215726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.216497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.218414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.222883 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.262091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1287/2572] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:28.263977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.275132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac49e1f-afc6-4f5d-953f-a14341cef94d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.335043 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.335595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.336000 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.336280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.337008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.338948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.366122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1286/2573] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:28.367375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.386128 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9c7ae-05c2-4876-8797-fcb9073d4fb6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.391203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.392568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.392860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.393141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.393888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.395518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.406491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1288/2574] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:28.410600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.421148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7dd72-115c-46f5-9d79-1d7d750e6a48 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.426358 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb347f3e-28c7-47c6-9125-4e7f2f9591e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.427041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb347f3e-28c7-47c6-9125-4e7f2f9591e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.427329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb347f3e-28c7-47c6-9125-4e7f2f9591e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.427610 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb347f3e-28c7-47c6-9125-4e7f2f9591e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.428407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb347f3e-28c7-47c6-9125-4e7f2f9591e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.430052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb347f3e-28c7-47c6-9125-4e7f2f9591e0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.442437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1287/2575] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:28.446600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.458848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f285ba-051d-4808-9547-a2cf11a22546 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.463480 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.464401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.464689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.464967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] PATH_INFO: `/v3/projects/9c2ab483b12349f88b4ce25775346610/users/c84514ea1d4f46cf967f2a189fae3417/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.481592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c2ab483b12349f88b4ce25775346610, user_id=c84514ea1d4f46cf967f2a189fae3417, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.484469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.497487 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1289/2576] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:28 2022] PUT /identity/v3/projects/9c2ab483b12349f88b4ce25775346610/users/c84514ea1d4f46cf967f2a189fae3417/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:28.507955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d1cb0-4cde-4c1c-baf7-52e78f25a5b4 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.508253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d1cb0-4cde-4c1c-baf7-52e78f25a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.508539 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d1cb0-4cde-4c1c-baf7-52e78f25a5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.536918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e63d1cb0-4cde-4c1c-baf7-52e78f25a5b4 None None] MFA Rules not processed for user `c84514ea1d4f46cf967f2a189fae3417`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.538170 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e63d1cb0-4cde-4c1c-baf7-52e78f25a5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.558645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.612222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1288/2577] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:28.616632 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.677076 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca001fec-98f3-43cd-9051-413ad0ad1a48 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.681720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.682373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.682663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.682944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.683634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.685585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.710370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1290/2578] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:28.714651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d1cb0-4cde-4c1c-baf7-52e78f25a5b4 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.725308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d1cb0-4cde-4c1c-baf7-52e78f25a5b4 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.780995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.781656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.781938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.782214 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.782908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.784850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.804379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1289/2579] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:28.808261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.819330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6fd80-38b4-4eaa-957b-f1fc32a30ed3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.824108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.824878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.825102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.825378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.826177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.827953 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.839460 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1291/2580] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:28.843298 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.853098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e85d-d824-463a-95c5-9b41ca561614 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.857694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea63a8-5b06-4e20-ae64-6cf598177ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.858352 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea63a8-5b06-4e20-ae64-6cf598177ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.858638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea63a8-5b06-4e20-ae64-6cf598177ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.858917 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea63a8-5b06-4e20-ae64-6cf598177ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.859653 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea63a8-5b06-4e20-ae64-6cf598177ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.861769 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea63a8-5b06-4e20-ae64-6cf598177ae8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.872038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1290/2581] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:28.876952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.886817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4259df-fcad-490d-8f13-2f00bf366755 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.891499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.892429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.892716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.892993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] PATH_INFO: `/v3/projects/c795f00462b24eeba59cf4d1509fa81d/users/345cb17ab4814d308e2ac063e4028504/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.908799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c795f00462b24eeba59cf4d1509fa81d, user_id=345cb17ab4814d308e2ac063e4028504, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.911585 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.926654 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1292/2582] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:28 2022] PUT /identity/v3/projects/c795f00462b24eeba59cf4d1509fa81d/users/345cb17ab4814d308e2ac063e4028504/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:28.941301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b481b6-842b-437a-b9c4-74262d49e397 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.941615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b481b6-842b-437a-b9c4-74262d49e397 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.942058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b481b6-842b-437a-b9c4-74262d49e397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.968563 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-72b481b6-842b-437a-b9c4-74262d49e397 None None] MFA Rules not processed for user `345cb17ab4814d308e2ac063e4028504`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.970302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-72b481b6-842b-437a-b9c4-74262d49e397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.038453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1291/2583] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:29.043319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.103127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.118579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b0e23-86c4-49a9-a0b5-22ea4c101e4c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.126147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.126832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.127201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.127502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.128249 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.130229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.163485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1293/2584] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:29.167860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b481b6-842b-437a-b9c4-74262d49e397 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.190014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b481b6-842b-437a-b9c4-74262d49e397 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.236367 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.271441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.272161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.272446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.272809 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.273535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.275838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.303864 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1292/2585] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:29.307363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.323961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec01ff-d8f6-445b-9f72-f1a8d3bbf178 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.333651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.334525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.335325 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.335610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.336675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.338025 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.355975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1294/2586] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:29.360183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.371339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ea18-ee57-4032-9738-fb5ea4e4329f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.377547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.378650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.379405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.379405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] PATH_INFO: `/v3/projects/64710cb530684d098e736e513ef2f3c1/users/fb69527cf7b54a10af9e8697150e24b2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.401434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64710cb530684d098e736e513ef2f3c1, user_id=fb69527cf7b54a10af9e8697150e24b2, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.405054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.422285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1293/2587] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:29 2022] PUT /identity/v3/projects/64710cb530684d098e736e513ef2f3c1/users/fb69527cf7b54a10af9e8697150e24b2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:29.426456 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.438099 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec3b15-4d7d-45dd-9577-a7d38e9e8081 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.511449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf3e0b-b2c9-4970-bd32-351dbcb42a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.519776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf3e0b-b2c9-4970-bd32-351dbcb42a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.520967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf3e0b-b2c9-4970-bd32-351dbcb42a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.521346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf3e0b-b2c9-4970-bd32-351dbcb42a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.522373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf3e0b-b2c9-4970-bd32-351dbcb42a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.524354 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf3e0b-b2c9-4970-bd32-351dbcb42a19 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.539786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1295/2588] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:29.544218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.554795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8ca22-92da-40b7-a210-ab766c846a3c admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.559491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.560431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.560717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.561019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] PATH_INFO: `/v3/projects/64710cb530684d098e736e513ef2f3c1/users/fb69527cf7b54a10af9e8697150e24b2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.563070 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64710cb530684d098e736e513ef2f3c1, user_id=fb69527cf7b54a10af9e8697150e24b2, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.566181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.577234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1294/2589] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:29 2022] PUT /identity/v3/projects/64710cb530684d098e736e513ef2f3c1/users/fb69527cf7b54a10af9e8697150e24b2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:29.591192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8238a82-dff3-4b0f-a88b-558e4858cddd None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.591483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8238a82-dff3-4b0f-a88b-558e4858cddd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.591762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8238a82-dff3-4b0f-a88b-558e4858cddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.632627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a8238a82-dff3-4b0f-a88b-558e4858cddd None None] MFA Rules not processed for user `fb69527cf7b54a10af9e8697150e24b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:29.633962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a8238a82-dff3-4b0f-a88b-558e4858cddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.730223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1296/2590] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:29.740568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.742562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.823384 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3371ac8-7573-415e-8845-a58bc3879151 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.831754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.832762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.833120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.833502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.932874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.934394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.946734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1295/2591] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:31.483374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8238a82-dff3-4b0f-a88b-558e4858cddd None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.493791 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8238a82-dff3-4b0f-a88b-558e4858cddd None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.497452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.498755 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.499112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.499452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.547724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:31.565409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.566924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.576413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1297/2592] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:32.078042 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.080076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:32.091143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0ce82-883a-41fa-ad43-16c279e31ab5 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.096585 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.097234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.097516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.097794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.109528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.109528 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.118511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1296/2593] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:33.352629 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.354027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:33.425081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a46f5-224f-413f-98cc-333119265613 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.429826 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ebfcb-2543-46a2-ab19-f5aa83330315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.430478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ebfcb-2543-46a2-ab19-f5aa83330315 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.430761 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ebfcb-2543-46a2-ab19-f5aa83330315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.431091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ebfcb-2543-46a2-ab19-f5aa83330315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.433169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-300ebfcb-2543-46a2-ab19-f5aa83330315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:33.500862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ebfcb-2543-46a2-ab19-f5aa83330315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.502830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ebfcb-2543-46a2-ab19-f5aa83330315 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.513205 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1298/2594] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:36.113016 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.124461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63e4c0-ed45-4551-ab3d-b247e93bafc1 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.130186 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.131059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.131347 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.131628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] PATH_INFO: `/v3/users/c84514ea1d4f46cf967f2a189fae3417` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.132854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84514ea1d4f46cf967f2a189fae3417)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.134996 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.157815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.225592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84514ea1d4f46cf967f2a189fae3417'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:36.229424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84514ea1d4f46cf967f2a189fae3417'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:36.230822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84514ea1d4f46cf967f2a189fae3417'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:36.266133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84514ea1d4f46cf967f2a189fae3417'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:36.279597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84514ea1d4f46cf967f2a189fae3417'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:36.299498 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1297/2595] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:36 2022] DELETE /identity/v3/users/c84514ea1d4f46cf967f2a189fae3417 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:36.304225 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.305093 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.305563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.324901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:36.326498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:36.397286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1299/2596] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:36.407077 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.409023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:36.485334 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d3610-adb7-4dc7-b681-0b71f0c91eb1 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.490169 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.490972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.491512 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.491810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.503881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.505733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.518050 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1298/2597] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:37.012884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.015227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.044783 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd5721-12da-4a67-9b67-64be559784c9 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.051517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.052437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.052709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.052980 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] PATH_INFO: `/v3/projects/9c2ab483b12349f88b4ce25775346610` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.054171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c2ab483b12349f88b4ce25775346610)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.056130 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.066463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.068907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.104358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.124887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c2ab483b12349f88b4ce25775346610', 'request_id': 'req-9c97a16b-462c-4f80-9c97-a452053a23ac'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.139080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c2ab483b12349f88b4ce25775346610', 'request_id': 'req-9c97a16b-462c-4f80-9c97-a452053a23ac'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.153580 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c2ab483b12349f88b4ce25775346610', 'request_id': 'req-9c97a16b-462c-4f80-9c97-a452053a23ac'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.168123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] The token cache is being invalidate because project 9c2ab483b12349f88b4ce25775346610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:37.168571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.169495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.169850 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d25a31-3351-4c4d-8bba-82f371f038f3 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.171435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1300/2598] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:37 2022] DELETE /identity/v3/projects/9c2ab483b12349f88b4ce25775346610 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:37.172824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.174819 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.227223 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.255634 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.256311 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.256595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.256724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97a16b-462c-4f80-9c97-a452053a23ac admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.256872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.258629 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.261352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.262260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.262544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.262993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] PATH_INFO: `/v3/users/b34132c9382a44cd97bc2c4e40c97eec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.264274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b34132c9382a44cd97bc2c4e40c97eec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.266244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.270315 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.271899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.281728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1299/2599] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:37.282558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.283916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.297914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f44f-8d70-4cb6-a26f-84376c16e288 service placement] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.303141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.304005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.304306 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.304564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] PATH_INFO: `/v3/users/345cb17ab4814d308e2ac063e4028504` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.305908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=345cb17ab4814d308e2ac063e4028504)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.307916 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.323972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b34132c9382a44cd97bc2c4e40c97eec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.324670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b34132c9382a44cd97bc2c4e40c97eec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.333186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b34132c9382a44cd97bc2c4e40c97eec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.333915 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b34132c9382a44cd97bc2c4e40c97eec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.348172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b34132c9382a44cd97bc2c4e40c97eec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.359060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '345cb17ab4814d308e2ac063e4028504'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.359778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '345cb17ab4814d308e2ac063e4028504'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.360465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '345cb17ab4814d308e2ac063e4028504'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.364866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1301/2600] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:37 2022] DELETE /identity/v3/users/b34132c9382a44cd97bc2c4e40c97eec => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:37.376861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '345cb17ab4814d308e2ac063e4028504'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.384096 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '345cb17ab4814d308e2ac063e4028504'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.397730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1300/2601] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:37 2022] DELETE /identity/v3/users/345cb17ab4814d308e2ac063e4028504 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:37.585184 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.587284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.640329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.676966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a83e5c-b18f-4a79-94fd-923a4da3c0e9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.696986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.698572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.748790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.749682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.749973 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.750251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] PATH_INFO: `/v3/projects/b114bd896cc6469184097ef9300613da` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.751469 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b114bd896cc6469184097ef9300613da)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.753437 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.771592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb152e33-cc30-4816-bcd4-27141c111b6c admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.776732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.777597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.777878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.778153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] PATH_INFO: `/v3/projects/c795f00462b24eeba59cf4d1509fa81d` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.779458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c795f00462b24eeba59cf4d1509fa81d)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.781459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.806272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b114bd896cc6469184097ef9300613da', 'request_id': 'req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.813569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b114bd896cc6469184097ef9300613da', 'request_id': 'req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.819837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b114bd896cc6469184097ef9300613da', 'request_id': 'req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.830161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.834410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] The token cache is being invalidate because project b114bd896cc6469184097ef9300613da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:37.834838 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.835497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-08cc7b14-3c93-45bd-be9f-2d9bd70c657e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.835972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c795f00462b24eeba59cf4d1509fa81d', 'request_id': 'req-f512e29c-974a-48bc-a4da-a2e9737221de'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.837306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1302/2602] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:37 2022] DELETE /identity/v3/projects/b114bd896cc6469184097ef9300613da => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:37.842282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806afb4c-57d5-4714-8bde-c2952667623e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.842770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806afb4c-57d5-4714-8bde-c2952667623e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.843121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806afb4c-57d5-4714-8bde-c2952667623e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.844420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c795f00462b24eeba59cf4d1509fa81d', 'request_id': 'req-f512e29c-974a-48bc-a4da-a2e9737221de'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.851154 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c795f00462b24eeba59cf4d1509fa81d', 'request_id': 'req-f512e29c-974a-48bc-a4da-a2e9737221de'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.862000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-806afb4c-57d5-4714-8bde-c2952667623e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:37.862150 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] The token cache is being invalidate because project c795f00462b24eeba59cf4d1509fa81d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:37.862284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.862929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:37.863309 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-806afb4c-57d5-4714-8bde-c2952667623e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.864578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1301/2603] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:37 2022] DELETE /identity/v3/projects/c795f00462b24eeba59cf4d1509fa81d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:37.868415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.869656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.924890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1303/2604] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:37.932355 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdc724-c9af-4c36-9c68-8c5f8ef9b61b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.932583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdc724-c9af-4c36-9c68-8c5f8ef9b61b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.932862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdc724-c9af-4c36-9c68-8c5f8ef9b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.944722 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512e29c-974a-48bc-a4da-a2e9737221de admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.949394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.950273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.950561 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.950843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] PATH_INFO: `/v3/users/fb69527cf7b54a10af9e8697150e24b2` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.951407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f6bdc724-c9af-4c36-9c68-8c5f8ef9b61b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:37.952112 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb69527cf7b54a10af9e8697150e24b2)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.952617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f6bdc724-c9af-4c36-9c68-8c5f8ef9b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:37.954178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.961030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1304/2605] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:37.965138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdc724-c9af-4c36-9c68-8c5f8ef9b61b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.974264 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdc724-c9af-4c36-9c68-8c5f8ef9b61b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.978942 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.979700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.980102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.980389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.981302 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:37.981622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.983671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.987312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.999929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1305/2606] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:38.004261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.006084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb69527cf7b54a10af9e8697150e24b2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.006784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb69527cf7b54a10af9e8697150e24b2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.007405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb69527cf7b54a10af9e8697150e24b2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.014135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace073e-1b3b-43b1-aba5-50125017e551 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.018807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.019446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.019715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb69527cf7b54a10af9e8697150e24b2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.019873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.020033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.020727 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.022633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.026687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb69527cf7b54a10af9e8697150e24b2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.042242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1302/2607] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:30:37 2022] DELETE /identity/v3/users/fb69527cf7b54a10af9e8697150e24b2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:38.053958 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1306/2608] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:30:38 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:38.057858 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.059275 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:38.118645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:38.133041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75090c-0f9a-4da8-968e-2b9744d8b848 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.193490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.194167 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.194450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.194727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.195433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.197888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.221981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1303/2609] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:30:38 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:38.225876 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.236865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a641e38-24a7-4478-a754-0df4b18627e8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.241901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.242779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.243069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.243659 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.244135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.245744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.256048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1307/2610] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:38.260035 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.270474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393c814-b93d-40b2-b6a3-221aa09441d8 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.275165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.275849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.276155 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.276833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.277166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.278776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.292370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1304/2611] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:38.293568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.304760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb33838-f32b-465f-b9de-9ee92ef32bb3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.315234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6964d2-3471-4954-86fc-a8411784dc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.316495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6964d2-3471-4954-86fc-a8411784dc79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.316782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6964d2-3471-4954-86fc-a8411784dc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.317095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6964d2-3471-4954-86fc-a8411784dc79 admin admin] PATH_INFO: `/v3/projects/6a6c40fbca064c789708300a00c36ce4/users/4bf0025854d64392810c6c9303eb9320/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.328639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.331418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:38.334298 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6964d2-3471-4954-86fc-a8411784dc79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6c40fbca064c789708300a00c36ce4, user_id=4bf0025854d64392810c6c9303eb9320, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.337823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6964d2-3471-4954-86fc-a8411784dc79 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.346005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a486e-21f9-4dc3-8ba4-0136a8277e30 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.353808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1308/2612] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:30:38 2022] PUT /identity/v3/projects/6a6c40fbca064c789708300a00c36ce4/users/4bf0025854d64392810c6c9303eb9320/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:38.358649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.358948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.359217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.377724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:38.378930 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:38.407752 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.408742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.409099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.409387 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] PATH_INFO: `/v3/projects/64710cb530684d098e736e513ef2f3c1` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.410597 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64710cb530684d098e736e513ef2f3c1)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.412793 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.442839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1309/2613] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:38.453711 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.455252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:38.463260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64710cb530684d098e736e513ef2f3c1', 'request_id': 'req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.470991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64710cb530684d098e736e513ef2f3c1', 'request_id': 'req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.479614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64710cb530684d098e736e513ef2f3c1', 'request_id': 'req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.493091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] The token cache is being invalidate because project 64710cb530684d098e736e513ef2f3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:38.493371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.494109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c8f07ce0-6772-4cef-a3a8-13d7e0044b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:38.495714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1305/2614] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:30:38 2022] DELETE /identity/v3/projects/64710cb530684d098e736e513ef2f3c1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:38.520143 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6aee5-9538-4be6-a07c-d29fb3eae94b None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.576600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.577266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.577548 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.577825 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.579796 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:38.590255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:38.656077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.657633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.668103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1310/2615] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:42.577491 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28089fa5-9874-422d-a713-ae89ddbfe8e0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:42.578089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28089fa5-9874-422d-a713-ae89ddbfe8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:42.578089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28089fa5-9874-422d-a713-ae89ddbfe8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:42.596603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:42.607941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-28089fa5-9874-422d-a713-ae89ddbfe8e0 None None] MFA Rules not processed for user `4bf0025854d64392810c6c9303eb9320`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:42.609527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-28089fa5-9874-422d-a713-ae89ddbfe8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:42.676270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1306/2616] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:42.690557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:42.692720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:42.768634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435126d-e607-4644-961e-e2ae78c12577 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:42.774113 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:42.774776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:42.775060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:42.775367 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:42.784308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:42.785910 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:42.794957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1311/2617] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:44.225580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:44.226084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:44.226203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:44.244904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] MFA Rules not processed for user `4bf0025854d64392810c6c9303eb9320`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:44.246116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:44.254141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1307/2618] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:30:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:44.562932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:44.565311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:44.578525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa767f0-c20a-44ea-822b-216e4ed55ac7 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:44.582135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:44.582787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:44.583072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:44.583350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:44.585605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:44.596598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:44.598021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:44.605873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1312/2619] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:30:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:54.239323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=31 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:54.602304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=65 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:06.771474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:06.772873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:06.784478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b5dc-1057-4397-821b-d84800fb3a4c None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:06.788051 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:06.788698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:06.788982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:06.789262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:06.791058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:06.803608 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:06.805092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:06.816244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1308/2620] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:09.980158 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.981651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:09.996832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803e484-f3d2-4da2-b316-e6d22e6b8b0f service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.001971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.002843 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.003132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.003412 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] PATH_INFO: `/v3/users/720bd38555404aafa93db5ae49b13acf` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.004650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] RBAC: Authorizing `identity:delete_user(user_id=720bd38555404aafa93db5ae49b13acf)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.006579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.052861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720bd38555404aafa93db5ae49b13acf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.053425 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720bd38555404aafa93db5ae49b13acf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.060358 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720bd38555404aafa93db5ae49b13acf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.060978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720bd38555404aafa93db5ae49b13acf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.073109 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-bad3d91a-397a-402f-8bfe-6adddb64497b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720bd38555404aafa93db5ae49b13acf'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.085575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1313/2621] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:31:09 2022] DELETE /identity/v3/users/720bd38555404aafa93db5ae49b13acf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:10.266363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.277045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc3df-3250-407c-ac38-421fb34fa74e service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.334632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.335490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.335775 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.336092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] PATH_INFO: `/v3/projects/3931210010614a1f987661568f29e851` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.337271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3931210010614a1f987661568f29e851)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.339178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.384309 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3931210010614a1f987661568f29e851', 'request_id': 'req-d3020548-38e7-4054-88e1-af417cbf7619'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.391021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3931210010614a1f987661568f29e851', 'request_id': 'req-d3020548-38e7-4054-88e1-af417cbf7619'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.396420 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3931210010614a1f987661568f29e851', 'request_id': 'req-d3020548-38e7-4054-88e1-af417cbf7619'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.406927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] The token cache is being invalidate because project 3931210010614a1f987661568f29e851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:10.407212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.408366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-d3020548-38e7-4054-88e1-af417cbf7619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.409883 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1309/2622] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:31:10 2022] DELETE /identity/v3/projects/3931210010614a1f987661568f29e851 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:10.414955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb23ea-cf8a-4ae1-8313-9eebf5d38971 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.415250 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb23ea-cf8a-4ae1-8313-9eebf5d38971 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.415534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb23ea-cf8a-4ae1-8313-9eebf5d38971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.433301 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-a7bb23ea-cf8a-4ae1-8313-9eebf5d38971 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:10.434511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a7bb23ea-cf8a-4ae1-8313-9eebf5d38971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.493677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1314/2623] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:10.501668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dab3bb-88e7-4619-8ff3-24d3d66ccc26 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.501954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dab3bb-88e7-4619-8ff3-24d3d66ccc26 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.502256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dab3bb-88e7-4619-8ff3-24d3d66ccc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.521038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-90dab3bb-88e7-4619-8ff3-24d3d66ccc26 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:10.522246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-90dab3bb-88e7-4619-8ff3-24d3d66ccc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.530131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1310/2624] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:10.534935 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb23ea-cf8a-4ae1-8313-9eebf5d38971 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.544615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb23ea-cf8a-4ae1-8313-9eebf5d38971 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.549616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.550350 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.550635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.550913 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.551639 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:10.551979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.553576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.565433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1315/2625] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:31:10.569745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dab3bb-88e7-4619-8ff3-24d3d66ccc26 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.578949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dab3bb-88e7-4619-8ff3-24d3d66ccc26 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.584211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.584915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.585144 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.585425 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.586114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.588045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.613226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1311/2626] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:10.617189 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.626937 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17798c-4e4b-4db8-a9ff-c1d3ac7d377c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.681334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.681992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.682280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.682583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.683279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.685213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.691989 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:10.706405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1316/2627] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:10.710567 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.721327 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616dd3d-8d51-44e6-8771-f43232c33aaf admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.725967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.726618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.726903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.727182 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.727994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.729603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.739805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1312/2628] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:10.743513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.753193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e4483-6545-4cc7-b977-e0bde666369c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.757670 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d5a91-1eaa-4eac-b9b3-ae550bad7b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.758311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5a91-1eaa-4eac-b9b3-ae550bad7b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.758594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5a91-1eaa-4eac-b9b3-ae550bad7b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.758873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5a91-1eaa-4eac-b9b3-ae550bad7b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.759812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d5a91-1eaa-4eac-b9b3-ae550bad7b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.761444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d5a91-1eaa-4eac-b9b3-ae550bad7b70 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.771570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1317/2629] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:10.775464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.787103 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf47ebd-e06a-4973-90f6-f3b7bca81976 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.791679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.792583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.793733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.793733 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] PATH_INFO: `/v3/projects/4c7bdda3904f4ea0ace6d2da550dc8a3/users/46bbc9e12b4847a5a2dc88f75116134a/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.807976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7bdda3904f4ea0ace6d2da550dc8a3, user_id=46bbc9e12b4847a5a2dc88f75116134a, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.810633 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.824497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1313/2630] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:31:10 2022] PUT /identity/v3/projects/4c7bdda3904f4ea0ace6d2da550dc8a3/users/46bbc9e12b4847a5a2dc88f75116134a/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:10.837234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628f3e-1614-4a57-9da4-2dd8ea1728e7 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.837463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628f3e-1614-4a57-9da4-2dd8ea1728e7 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.837752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628f3e-1614-4a57-9da4-2dd8ea1728e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.863682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-ac628f3e-1614-4a57-9da4-2dd8ea1728e7 None None] MFA Rules not processed for user `46bbc9e12b4847a5a2dc88f75116134a`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:10.864971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ac628f3e-1614-4a57-9da4-2dd8ea1728e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.929503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1318/2631] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:10.934189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.948773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:11.005289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b388dd-3ed7-4cd7-99a9-f6f57d21efde admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.010083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.010744 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.011029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.011308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.012049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.013987 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.044872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1314/2632] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:11.046018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628f3e-1614-4a57-9da4-2dd8ea1728e7 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.056815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628f3e-1614-4a57-9da4-2dd8ea1728e7 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.113405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.114090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.114383 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.114673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.115381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.117410 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.137058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1319/2633] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:31:11 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:11.141033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.151800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902761bd-befa-4a32-8308-8ca71ce1db1f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.156450 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.157098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.157437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.157716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.158473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.160138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.170529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1315/2634] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:11.175049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.185523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641cbb-457f-4d11-a676-05a5b1d3f634 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.191118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.192242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.192525 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.192797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] PATH_INFO: `/v3/projects/86c2336189394e64a72fd7afad8fdeb2/users/eda19c78b5e543ff9e89867c1b650eec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.208067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c2336189394e64a72fd7afad8fdeb2, user_id=eda19c78b5e543ff9e89867c1b650eec, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.216592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.229885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1320/2635] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:31:11 2022] PUT /identity/v3/projects/86c2336189394e64a72fd7afad8fdeb2/users/eda19c78b5e543ff9e89867c1b650eec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:11.233997 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.244603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb60d-239e-4fc8-a2d0-2f7e9df3cd96 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.299076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368f7ef-b50c-47c8-ba3f-2680e202b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.299758 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f7ef-b50c-47c8-ba3f-2680e202b5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.300023 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f7ef-b50c-47c8-ba3f-2680e202b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.300299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f7ef-b50c-47c8-ba3f-2680e202b5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.301018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368f7ef-b50c-47c8-ba3f-2680e202b5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.302604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368f7ef-b50c-47c8-ba3f-2680e202b5d0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.312105 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1316/2636] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:11.317102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.327352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe5a8d-94a2-4a30-9946-8c33d9580f46 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.332050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.332945 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.333218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.333501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] PATH_INFO: `/v3/projects/86c2336189394e64a72fd7afad8fdeb2/users/eda19c78b5e543ff9e89867c1b650eec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.335496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c2336189394e64a72fd7afad8fdeb2, user_id=eda19c78b5e543ff9e89867c1b650eec, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.338227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.348344 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1321/2637] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:31:11 2022] PUT /identity/v3/projects/86c2336189394e64a72fd7afad8fdeb2/users/eda19c78b5e543ff9e89867c1b650eec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:11.358181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0afdf0-6344-4413-ba65-0260b2094af9 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.358502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0afdf0-6344-4413-ba65-0260b2094af9 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.358787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0afdf0-6344-4413-ba65-0260b2094af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.385389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-dd0afdf0-6344-4413-ba65-0260b2094af9 None None] MFA Rules not processed for user `eda19c78b5e543ff9e89867c1b650eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:11.386586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-dd0afdf0-6344-4413-ba65-0260b2094af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:11.457741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1317/2638] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:11.468752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.470531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:11.493177 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:11.529098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1eea27-300d-46a6-b4ad-3bef3ebe3c40 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.533544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1af0d-cd9e-457e-8e45-5baf6b27d072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.534198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1af0d-cd9e-457e-8e45-5baf6b27d072 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.534491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1af0d-cd9e-457e-8e45-5baf6b27d072 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.534865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1af0d-cd9e-457e-8e45-5baf6b27d072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.544494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1af0d-cd9e-457e-8e45-5baf6b27d072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.545993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1af0d-cd9e-457e-8e45-5baf6b27d072 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.556027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1322/2639] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:11.937418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0afdf0-6344-4413-ba65-0260b2094af9 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.947382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:12.002895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0afdf0-6344-4413-ba65-0260b2094af9 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.007397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.008300 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.008581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.008855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] PATH_INFO: `/v3/users/46bbc9e12b4847a5a2dc88f75116134a` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.010034 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46bbc9e12b4847a5a2dc88f75116134a)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.012110 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.059310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46bbc9e12b4847a5a2dc88f75116134a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.060037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46bbc9e12b4847a5a2dc88f75116134a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.060667 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46bbc9e12b4847a5a2dc88f75116134a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.075632 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46bbc9e12b4847a5a2dc88f75116134a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.083506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46bbc9e12b4847a5a2dc88f75116134a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.098219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1318/2640] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:31:11 2022] DELETE /identity/v3/users/46bbc9e12b4847a5a2dc88f75116134a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:12.102799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.103280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.103502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.121010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:12.122294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:12.199762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1323/2641] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:12.210446 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.211910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:12.284472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f6ed6-8e0d-49d5-b976-4eb837f499d1 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.288160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.288810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.289093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.289369 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.298645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.300114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.309382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1319/2642] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:12.661279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.662854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:12.691059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa749cee-6c63-4414-a7a0-1f194889caa8 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.696554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.697606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.698509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.698866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] PATH_INFO: `/v3/projects/4c7bdda3904f4ea0ace6d2da550dc8a3` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.700217 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7bdda3904f4ea0ace6d2da550dc8a3)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.702428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.756841 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7bdda3904f4ea0ace6d2da550dc8a3', 'request_id': 'req-44a006bd-7514-41bf-ac93-16cb75e989b4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.763162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7bdda3904f4ea0ace6d2da550dc8a3', 'request_id': 'req-44a006bd-7514-41bf-ac93-16cb75e989b4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.768867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7bdda3904f4ea0ace6d2da550dc8a3', 'request_id': 'req-44a006bd-7514-41bf-ac93-16cb75e989b4'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.783007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] The token cache is being invalidate because project 4c7bdda3904f4ea0ace6d2da550dc8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:12.783652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.784456 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.786013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1324/2643] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:31:12 2022] DELETE /identity/v3/projects/4c7bdda3904f4ea0ace6d2da550dc8a3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:12.790094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.801183 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f811a-8b80-42b5-b856-fb4d92e1d941 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.844458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:12.878098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.879344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.879344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.879583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] PATH_INFO: `/v3/users/eda19c78b5e543ff9e89867c1b650eec` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.880875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda19c78b5e543ff9e89867c1b650eec)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.882773 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.944308 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda19c78b5e543ff9e89867c1b650eec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.945059 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda19c78b5e543ff9e89867c1b650eec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.945687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda19c78b5e543ff9e89867c1b650eec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.958588 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda19c78b5e543ff9e89867c1b650eec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.969233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-42e80d91-8772-46b1-9199-cd99372567ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda19c78b5e543ff9e89867c1b650eec'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:12.983027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1320/2644] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:31:12 2022] DELETE /identity/v3/users/eda19c78b5e543ff9e89867c1b650eec => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:13.270461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.272027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:13.297811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a006bd-7514-41bf-ac93-16cb75e989b4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.393056 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:13.413892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.414888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.415234 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.415458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] PATH_INFO: `/v3/projects/86c2336189394e64a72fd7afad8fdeb2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.417050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c2336189394e64a72fd7afad8fdeb2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.418978 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.472874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c2336189394e64a72fd7afad8fdeb2', 'request_id': 'req-c200a9a1-b054-47c9-8e20-4d30548a7c0c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:13.479956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c2336189394e64a72fd7afad8fdeb2', 'request_id': 'req-c200a9a1-b054-47c9-8e20-4d30548a7c0c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:13.485939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c2336189394e64a72fd7afad8fdeb2', 'request_id': 'req-c200a9a1-b054-47c9-8e20-4d30548a7c0c'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:13.498605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] The token cache is being invalidate because project 86c2336189394e64a72fd7afad8fdeb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:13.499102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:13.499806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-c200a9a1-b054-47c9-8e20-4d30548a7c0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:13.501716 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1325/2645] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:31:13 2022] DELETE /identity/v3/projects/86c2336189394e64a72fd7afad8fdeb2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:13.507171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a4fe-ce33-4815-8f4b-b29b593a16e5 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.507687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a4fe-ce33-4815-8f4b-b29b593a16e5 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.507788 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a4fe-ce33-4815-8f4b-b29b593a16e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.525871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-a2a9a4fe-ce33-4815-8f4b-b29b593a16e5 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:13.527085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a2a9a4fe-ce33-4815-8f4b-b29b593a16e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:13.597184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1321/2646] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:13.606618 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff2c2-0185-424b-bccb-049fa9eb715c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.606914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff2c2-0185-424b-bccb-049fa9eb715c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.607199 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff2c2-0185-424b-bccb-049fa9eb715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.626427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d59ff2c2-0185-424b-bccb-049fa9eb715c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:13.627619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d59ff2c2-0185-424b-bccb-049fa9eb715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:13.635362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1326/2647] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:13.639499 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9a4fe-ce33-4815-8f4b-b29b593a16e5 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.649579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9a4fe-ce33-4815-8f4b-b29b593a16e5 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.654058 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.654700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.654983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.655261 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.656046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:13.656372 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.658628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.670292 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1322/2648] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:13.674221 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ff2c2-0185-424b-bccb-049fa9eb715c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.683173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ff2c2-0185-424b-bccb-049fa9eb715c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.688277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.688927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.689212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.689490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.690187 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.692136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.717373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1327/2649] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:13.721662 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.731477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78267163-6e55-4f76-a259-280dc7fce651 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.762794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:13.787580 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.788313 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.788603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.788878 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.789566 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.791481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.810036 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1323/2650] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:13.814840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.824902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc2b79-d5d2-486b-856e-e72a087178b6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.830682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.831339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.831697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.831967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.832698 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.834283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.844929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1328/2651] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:13.848840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.858681 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfd27b-a6df-4e0c-b0eb-fbc602601160 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.863267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d6b6f-69de-4553-808e-6f44f0785d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.864053 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d6b6f-69de-4553-808e-6f44f0785d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.864343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d6b6f-69de-4553-808e-6f44f0785d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.864621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d6b6f-69de-4553-808e-6f44f0785d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.865348 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d6b6f-69de-4553-808e-6f44f0785d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.866932 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d6b6f-69de-4553-808e-6f44f0785d1e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.877289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1324/2652] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:13.881269 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.890939 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bc098-d27d-469d-a557-da8d1d207135 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.896125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.897015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.897295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.897567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] PATH_INFO: `/v3/projects/44b194a418084a41918de62b02412297/users/b27f6d44835a4df195c70f3c4126aae0/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.911746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b194a418084a41918de62b02412297, user_id=b27f6d44835a4df195c70f3c4126aae0, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.914478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.926887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1329/2653] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:31:13 2022] PUT /identity/v3/projects/44b194a418084a41918de62b02412297/users/b27f6d44835a4df195c70f3c4126aae0/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:13.937235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea29d-6c3a-4313-8028-10072cad8d85 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.937706 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea29d-6c3a-4313-8028-10072cad8d85 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.937986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea29d-6c3a-4313-8028-10072cad8d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.964489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-19fea29d-6c3a-4313-8028-10072cad8d85 None None] MFA Rules not processed for user `b27f6d44835a4df195c70f3c4126aae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:13.965692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-19fea29d-6c3a-4313-8028-10072cad8d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:14.028685 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1325/2654] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:14.033253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.095720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2bdac-e906-4fda-a918-f3b643e94b7c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.100361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.101014 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.101414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.101699 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.102398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.104319 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.121649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:14.132804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1330/2655] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:31:14 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:14.133641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fea29d-6c3a-4313-8028-10072cad8d85 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.144923 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fea29d-6c3a-4313-8028-10072cad8d85 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.199799 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.200506 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.200789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.201064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.201751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.203657 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.223354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1326/2656] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:31:14 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:14.227253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.237656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b3d3-ff35-4724-89fa-d2a1208c3edd admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.242215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.242865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.243153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.243429 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.244197 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.245778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.257560 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1331/2657] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:14.261464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.271038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca8f4a-8b29-4f04-83b4-5b0ffda6b635 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.275518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.276416 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.276701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.276975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] PATH_INFO: `/v3/projects/e40bf7f6c1004f198c396d44e3521de0/users/1055856b643442ebba7eb39efc068519/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.291484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40bf7f6c1004f198c396d44e3521de0, user_id=1055856b643442ebba7eb39efc068519, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.294086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.306943 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1327/2658] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:31:14 2022] PUT /identity/v3/projects/e40bf7f6c1004f198c396d44e3521de0/users/1055856b643442ebba7eb39efc068519/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:14.310862 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.321603 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d626f-be6f-446a-b5bd-63ce03141d0d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.376955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89f34b-6a72-4390-bb8d-13dcf6015cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.377680 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89f34b-6a72-4390-bb8d-13dcf6015cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.377957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89f34b-6a72-4390-bb8d-13dcf6015cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.378262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89f34b-6a72-4390-bb8d-13dcf6015cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.378990 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89f34b-6a72-4390-bb8d-13dcf6015cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.380616 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89f34b-6a72-4390-bb8d-13dcf6015cc7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.390318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1332/2659] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:31:14.394452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.404736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d1cb2-dae2-4ce7-8e5a-26c862da3c35 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.409735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8708c9-e0ee-426b-85fc-e023690622c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.410613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8708c9-e0ee-426b-85fc-e023690622c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.410949 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8708c9-e0ee-426b-85fc-e023690622c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.411160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8708c9-e0ee-426b-85fc-e023690622c4 admin admin] PATH_INFO: `/v3/projects/e40bf7f6c1004f198c396d44e3521de0/users/1055856b643442ebba7eb39efc068519/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.413134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8708c9-e0ee-426b-85fc-e023690622c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40bf7f6c1004f198c396d44e3521de0, user_id=1055856b643442ebba7eb39efc068519, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.415829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8708c9-e0ee-426b-85fc-e023690622c4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.422072 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:14.428401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1328/2660] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:31:14 2022] PUT /identity/v3/projects/e40bf7f6c1004f198c396d44e3521de0/users/1055856b643442ebba7eb39efc068519/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:14.440624 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.440916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.441198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.447080 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.447370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.447656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.465033 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] MFA Rules not processed for user `5c8ad338eb7b47c18dcb86f287e28858`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:14.466221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:14.468275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] MFA Rules not processed for user `1055856b643442ebba7eb39efc068519`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:14.469430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:14.521477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1329/2661] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:14.538817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1333/2662] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:14.549911 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.551616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:14.609855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d855e6b-2a22-4472-a5a7-ce9f7b5bf080 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.613617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.614260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.614541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.614818 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.673376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.674873 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.683912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1330/2663] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:14.833803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:14.835414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:14.894164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836978f0-8e31-4d8b-baf9-a9ef5bf842e1 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:14.898050 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:14.898695 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:14.898979 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:14.899256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:14.901679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:14.911940 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:14.913372 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.927157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1334/2664] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:19.279526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:19.280988 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:19.292525 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bab0e5-acb4-49f4-95c2-2461b27d9666 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:19.296131 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:19.296779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.297066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.297466 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.307046 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:19.308526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:19.319089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1331/2665] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:24.924600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:29.314458 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=80 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:41.734155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:41.735650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:41.813227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc928d-3266-4c55-bb37-25c730b8ecdd service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:41.818765 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f185c8-e3c7-4f38-ae77-92e19525afa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:41.819482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f185c8-e3c7-4f38-ae77-92e19525afa9 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:41.819768 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f185c8-e3c7-4f38-ae77-92e19525afa9 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:41.820082 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f185c8-e3c7-4f38-ae77-92e19525afa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:41.821908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-34f185c8-e3c7-4f38-ae77-92e19525afa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:41.836112 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f185c8-e3c7-4f38-ae77-92e19525afa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:41.837714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f185c8-e3c7-4f38-ae77-92e19525afa9 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:41.848536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1335/2666] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:42.129385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.212350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fd8e-d5b4-40e7-a9b1-145adf433039 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.218205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.219724 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.220146 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.220526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] PATH_INFO: `/v3/users/b27f6d44835a4df195c70f3c4126aae0` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.222497 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b27f6d44835a4df195c70f3c4126aae0)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.224784 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.290935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b27f6d44835a4df195c70f3c4126aae0'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.291627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b27f6d44835a4df195c70f3c4126aae0'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.292270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b27f6d44835a4df195c70f3c4126aae0'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.306783 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b27f6d44835a4df195c70f3c4126aae0'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.314989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b27f6d44835a4df195c70f3c4126aae0'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.330329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1332/2667] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:31:42 2022] DELETE /identity/v3/users/b27f6d44835a4df195c70f3c4126aae0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:42.334739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.335179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.335565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.354352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:42.356155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.421830 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1336/2668] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:42.431546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.433082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.494803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192a5e2-0133-416d-9330-1fa4533e73b5 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.499190 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.500014 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.500370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.500731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.516684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.518335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.529940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1333/2669] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:42.824011 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.825708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.844508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2ec40-85e0-4d61-b4c0-e2c779d88162 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.849285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.850244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.850598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.851072 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] PATH_INFO: `/v3/projects/44b194a418084a41918de62b02412297` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.852430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b194a418084a41918de62b02412297)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.854787 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.911262 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b194a418084a41918de62b02412297', 'request_id': 'req-64f34ebd-20c4-4515-bb63-00421824193d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.918449 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b194a418084a41918de62b02412297', 'request_id': 'req-64f34ebd-20c4-4515-bb63-00421824193d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.926080 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b194a418084a41918de62b02412297', 'request_id': 'req-64f34ebd-20c4-4515-bb63-00421824193d'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.941164 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] The token cache is being invalidate because project 44b194a418084a41918de62b02412297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:42.941453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.942075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.943615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1337/2670] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:31:42 2022] DELETE /identity/v3/projects/44b194a418084a41918de62b02412297 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:42.947544 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.957546 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209dadc-452f-4f44-9893-4d4d16c96d4f service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.974938 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:43.020164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.020969 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.021274 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.021540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] PATH_INFO: `/v3/users/1055856b643442ebba7eb39efc068519` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.022727 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1055856b643442ebba7eb39efc068519)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.024693 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.070779 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1055856b643442ebba7eb39efc068519'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.071410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1055856b643442ebba7eb39efc068519'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.072038 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1055856b643442ebba7eb39efc068519'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.083467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1055856b643442ebba7eb39efc068519'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.090400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-674aff49-ae24-438e-a91a-9486b684db9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1055856b643442ebba7eb39efc068519'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.103251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1334/2671] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:31:42 2022] DELETE /identity/v3/users/1055856b643442ebba7eb39efc068519 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:43.368229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.369738 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:43.382828 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f34ebd-20c4-4515-bb63-00421824193d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.388069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:43.443615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.444515 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.444807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.445089 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] PATH_INFO: `/v3/projects/e40bf7f6c1004f198c396d44e3521de0` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.446334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40bf7f6c1004f198c396d44e3521de0)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.448290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.493909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40bf7f6c1004f198c396d44e3521de0', 'request_id': 'req-0c718c39-8062-4b41-a860-62218f8df007'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.500096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40bf7f6c1004f198c396d44e3521de0', 'request_id': 'req-0c718c39-8062-4b41-a860-62218f8df007'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.505576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40bf7f6c1004f198c396d44e3521de0', 'request_id': 'req-0c718c39-8062-4b41-a860-62218f8df007'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.517458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] The token cache is being invalidate because project e40bf7f6c1004f198c396d44e3521de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:43.517745 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.518365 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0c718c39-8062-4b41-a860-62218f8df007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.519896 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1338/2672] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:31:43 2022] DELETE /identity/v3/projects/e40bf7f6c1004f198c396d44e3521de0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:43.525500 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be59535-94f5-468e-9aa7-1cb2cef8f16f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.525980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be59535-94f5-468e-9aa7-1cb2cef8f16f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.526084 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be59535-94f5-468e-9aa7-1cb2cef8f16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.544022 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9be59535-94f5-468e-9aa7-1cb2cef8f16f None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:43.545166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9be59535-94f5-468e-9aa7-1cb2cef8f16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:43.603660 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1335/2673] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:43.611568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d9b9-1980-46d3-a9d9-4e798c4052a0 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.611890 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d9b9-1980-46d3-a9d9-4e798c4052a0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.612178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d9b9-1980-46d3-a9d9-4e798c4052a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.630279 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d723d9b9-1980-46d3-a9d9-4e798c4052a0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:43.631475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d723d9b9-1980-46d3-a9d9-4e798c4052a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:43.639576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1339/2674] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:43.644419 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be59535-94f5-468e-9aa7-1cb2cef8f16f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.653947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be59535-94f5-468e-9aa7-1cb2cef8f16f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.659127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.659776 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.660098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.660380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.661111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:43.661423 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.663012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.674312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1336/2675] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:43.678364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723d9b9-1980-46d3-a9d9-4e798c4052a0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.687717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723d9b9-1980-46d3-a9d9-4e798c4052a0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.692366 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.693007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.693295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.693574 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.694258 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.696178 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.721821 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1340/2676] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:43.725935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.735808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb60f9-74df-4d94-8e97-0356b59199cd admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.760984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:43.794453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.795052 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.795340 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.795622 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.796426 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.798344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.818475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1337/2677] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:43.822573 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.833567 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a103ca-396b-4323-a2d8-e1bee9325115 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.838398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.839042 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.839324 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.839601 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.840360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.841940 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.852396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1341/2678] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:43.856185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.865737 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57eb41-f3fe-455b-9a0c-c22c3f359324 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.870235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22794e70-b211-4e1b-83bf-6dc59365486e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.870879 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22794e70-b211-4e1b-83bf-6dc59365486e admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.871161 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22794e70-b211-4e1b-83bf-6dc59365486e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.871436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22794e70-b211-4e1b-83bf-6dc59365486e admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.872196 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22794e70-b211-4e1b-83bf-6dc59365486e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.873771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22794e70-b211-4e1b-83bf-6dc59365486e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.883853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1338/2679] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:43.887532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.897673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253f99-c14a-4dff-900e-72a286831934 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.902211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.903070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.903353 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.903623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] PATH_INFO: `/v3/projects/2e8bb0a7f00d46e6b619e1d821053481/users/d948b3ed7938421db26c0b53b75d0ee8/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.917769 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8bb0a7f00d46e6b619e1d821053481, user_id=d948b3ed7938421db26c0b53b75d0ee8, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.920427 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.933216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1342/2680] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:31:43 2022] PUT /identity/v3/projects/2e8bb0a7f00d46e6b619e1d821053481/users/d948b3ed7938421db26c0b53b75d0ee8/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:43.944286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.944579 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.944860 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.971551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] MFA Rules not processed for user `d948b3ed7938421db26c0b53b75d0ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:43.972763 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:44.033871 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1339/2681] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:44.044815 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.046981 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:44.086975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:44.106220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b1fd-628c-4a65-a40d-cd4df5aac18d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:44.110532 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:44.111216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.111505 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.111788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.120914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.122479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.132323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1343/2682] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:53.550143 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:53.551193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.618312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c235a97-4ebd-41ed-82f5-3c5e40bc686b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:53.623244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6993f8-027c-47d0-ab60-340116dcdf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:53.624400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6993f8-027c-47d0-ab60-340116dcdf3c service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.624686 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6993f8-027c-47d0-ab60-340116dcdf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.624958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6993f8-027c-47d0-ab60-340116dcdf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.630551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-2d6993f8-027c-47d0-ab60-340116dcdf3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.677516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:53.689674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6993f8-027c-47d0-ab60-340116dcdf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:53.691109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6993f8-027c-47d0-ab60-340116dcdf3c service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:53.699913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1340/2683] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:54.127751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=10 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:58.452220 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:58.468092 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513fa5e-470a-4913-98ac-46c796504b84 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:58.473977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:58.474627 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:58.474965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:58.475259 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:58.477285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:58.557952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:58.559671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:58.571702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1344/2684] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:59.788791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0343c-8d55-4e9d-b914-181d705f86c3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.790073 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0343c-8d55-4e9d-b914-181d705f86c3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.790556 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0343c-8d55-4e9d-b914-181d705f86c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.808992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-c9e0343c-8d55-4e9d-b914-181d705f86c3 None None] MFA Rules not processed for user `4bf0025854d64392810c6c9303eb9320`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:59.810827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c9e0343c-8d55-4e9d-b914-181d705f86c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:59.906907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1341/2685] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:31:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:05.667591 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:05.670804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:05.700885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df990a05-ca9b-4a73-9ed0-42dc7132be47 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:05.706708 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:05.708555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:05.709026 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:05.709479 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] PATH_INFO: `/v3/users/fcabfe4f1a6b41049ca2bd37d31759aa` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:05.710867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcabfe4f1a6b41049ca2bd37d31759aa)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:05.712974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:05.800929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcabfe4f1a6b41049ca2bd37d31759aa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:05.802445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcabfe4f1a6b41049ca2bd37d31759aa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:05.817880 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcabfe4f1a6b41049ca2bd37d31759aa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:05.818554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcabfe4f1a6b41049ca2bd37d31759aa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:05.834689 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcabfe4f1a6b41049ca2bd37d31759aa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:05.854141 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1345/2686] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:32:05 2022] DELETE /identity/v3/users/fcabfe4f1a6b41049ca2bd37d31759aa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:06.022542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0343c-8d55-4e9d-b914-181d705f86c3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:06.036541 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0343c-8d55-4e9d-b914-181d705f86c3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:06.105577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:06.106431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:06.106718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:06.106982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] PATH_INFO: `/v3/projects/1d73ce3d68ac4f6e99f9a3b194cde320` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:06.108186 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d73ce3d68ac4f6e99f9a3b194cde320)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:06.110122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:06.156444 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d73ce3d68ac4f6e99f9a3b194cde320', 'request_id': 'req-64cfefc8-a9c6-4015-9a17-02a5c4e97938'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.163417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d73ce3d68ac4f6e99f9a3b194cde320', 'request_id': 'req-64cfefc8-a9c6-4015-9a17-02a5c4e97938'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.169156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d73ce3d68ac4f6e99f9a3b194cde320', 'request_id': 'req-64cfefc8-a9c6-4015-9a17-02a5c4e97938'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.180032 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] The token cache is being invalidate because project 1d73ce3d68ac4f6e99f9a3b194cde320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:06.180318 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.180942 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.182474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1342/2687] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:32:06 2022] DELETE /identity/v3/projects/1d73ce3d68ac4f6e99f9a3b194cde320 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:06.186445 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:06.188126 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:06.200822 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209bcb2-44bc-4d30-96d6-e1cb214713e6 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:06.255387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:06.256282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:06.256561 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:06.256837 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] PATH_INFO: `/v3/users/3d7b267cc708479080fb6e6c804a3dad` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:06.258028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7b267cc708479080fb6e6c804a3dad)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:06.259977 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:06.265706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:06.306807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7b267cc708479080fb6e6c804a3dad'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.307444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7b267cc708479080fb6e6c804a3dad'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.315223 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7b267cc708479080fb6e6c804a3dad'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.315871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7b267cc708479080fb6e6c804a3dad'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.328865 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7b267cc708479080fb6e6c804a3dad'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.343549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1346/2688] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:32:06 2022] DELETE /identity/v3/users/3d7b267cc708479080fb6e6c804a3dad => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:06.511242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:06.512688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:06.525798 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfefc8-a9c6-4015-9a17-02a5c4e97938 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:06.550173 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:06.585658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:06.586548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:06.586833 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:06.587111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] PATH_INFO: `/v3/projects/826d699dc70e4cf1a21d13dd9a4b61e3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:06.588386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=826d699dc70e4cf1a21d13dd9a4b61e3)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:06.590379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:06.638380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826d699dc70e4cf1a21d13dd9a4b61e3', 'request_id': 'req-41d984de-9c0a-4b09-b73c-396eb2cdb900'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.646123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826d699dc70e4cf1a21d13dd9a4b61e3', 'request_id': 'req-41d984de-9c0a-4b09-b73c-396eb2cdb900'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.651757 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826d699dc70e4cf1a21d13dd9a4b61e3', 'request_id': 'req-41d984de-9c0a-4b09-b73c-396eb2cdb900'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.662884 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] The token cache is being invalidate because project 826d699dc70e4cf1a21d13dd9a4b61e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:06.663169 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.663795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-41d984de-9c0a-4b09-b73c-396eb2cdb900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:06.665408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1343/2689] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:32:06 2022] DELETE /identity/v3/projects/826d699dc70e4cf1a21d13dd9a4b61e3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:16.339020 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:16.659927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:20.151127 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:20.152555 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:20.165726 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e507-c787-4263-a0ae-144798849349 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:20.219863 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:20.220721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:20.221007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:20.221926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] PATH_INFO: `/v3/users/d948b3ed7938421db26c0b53b75d0ee8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:20.223174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d948b3ed7938421db26c0b53b75d0ee8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:20.225118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:20.270291 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd948b3ed7938421db26c0b53b75d0ee8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.270936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd948b3ed7938421db26c0b53b75d0ee8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.277507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd948b3ed7938421db26c0b53b75d0ee8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.278255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd948b3ed7938421db26c0b53b75d0ee8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.290293 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd948b3ed7938421db26c0b53b75d0ee8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.302267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1347/2690] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:32:20 2022] DELETE /identity/v3/users/d948b3ed7938421db26c0b53b75d0ee8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:20.306831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:20.307323 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:20.307424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:20.324876 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:20.326045 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:20.384617 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1344/2691] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:20.396496 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:20.397988 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:20.454976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6356004-477b-4ff1-95f9-2de0f3ba1c71 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:20.459081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd512e2d-ff92-410f-aa6f-24cfac045777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:20.459720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd512e2d-ff92-410f-aa6f-24cfac045777 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:20.460027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd512e2d-ff92-410f-aa6f-24cfac045777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:20.460305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd512e2d-ff92-410f-aa6f-24cfac045777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:20.469125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd512e2d-ff92-410f-aa6f-24cfac045777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:20.470531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd512e2d-ff92-410f-aa6f-24cfac045777 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:20.481797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1348/2692] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:20.742422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:20.743874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:20.757094 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faea752-2237-4622-b8e1-a367ea999298 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:20.762018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:20.763204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:20.763587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:20.763967 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] PATH_INFO: `/v3/projects/2e8bb0a7f00d46e6b619e1d821053481` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:20.765269 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8bb0a7f00d46e6b619e1d821053481)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:20.767270 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:20.823020 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8bb0a7f00d46e6b619e1d821053481', 'request_id': 'req-ee8b1989-470b-4d75-868e-5fcbde81c6c8'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.830508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8bb0a7f00d46e6b619e1d821053481', 'request_id': 'req-ee8b1989-470b-4d75-868e-5fcbde81c6c8'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.837128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8bb0a7f00d46e6b619e1d821053481', 'request_id': 'req-ee8b1989-470b-4d75-868e-5fcbde81c6c8'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.848531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] The token cache is being invalidate because project 2e8bb0a7f00d46e6b619e1d821053481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:20.848822 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.849477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee8b1989-470b-4d75-868e-5fcbde81c6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:20.851108 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1345/2693] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:32:20 2022] DELETE /identity/v3/projects/2e8bb0a7f00d46e6b619e1d821053481 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:20.856253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46d33-76c5-4363-b845-4004110ee7fb None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:20.856565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46d33-76c5-4363-b845-4004110ee7fb None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:20.856846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46d33-76c5-4363-b845-4004110ee7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:20.873881 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-caf46d33-76c5-4363-b845-4004110ee7fb None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:20.875059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-caf46d33-76c5-4363-b845-4004110ee7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:20.891625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:20.935885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1349/2694] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:20.944363 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2452e-020f-487c-a67a-a4799e1ce92a None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:20.944649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2452e-020f-487c-a67a-a4799e1ce92a None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:20.944924 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2452e-020f-487c-a67a-a4799e1ce92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:20.962672 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-87e2452e-020f-487c-a67a-a4799e1ce92a None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:20.963887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-87e2452e-020f-487c-a67a-a4799e1ce92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:20.971624 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1346/2695] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:20.975592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf46d33-76c5-4363-b845-4004110ee7fb None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:20.985030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf46d33-76c5-4363-b845-4004110ee7fb None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:20.990022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:20.990661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:20.990943 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:20.991218 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:20.991969 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:20.992277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:20.993869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:21.006048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1350/2696] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:21.010083 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2452e-020f-487c-a67a-a4799e1ce92a None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:21.018975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2452e-020f-487c-a67a-a4799e1ce92a None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:21.024258 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:21.024893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:21.025175 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:21.025451 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:21.026160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:21.028066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:21.056281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1347/2697] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:32:21 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:32:21.058098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:21.068827 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00b35-9b37-4392-892d-fa8846f1d9de admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:21.122385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:21.123040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:21.123329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:21.123604 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:21.124323 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:21.126214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:21.147554 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1351/2698] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:32:21 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:32:21.151641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:21.161830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8a4c4-5e73-447f-973d-b2a313dc3817 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:21.166349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:21.166998 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:21.167284 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:21.167560 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:21.168350 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:21.169937 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:21.180540 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1348/2699] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:32:21.184146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:21.194638 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60777be3-de0a-45eb-83d2-0ca607bc6584 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:21.199069 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a85bb-b79b-498f-b6b1-983f01342097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:21.199706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a85bb-b79b-498f-b6b1-983f01342097 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:21.200034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a85bb-b79b-498f-b6b1-983f01342097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:21.200308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a85bb-b79b-498f-b6b1-983f01342097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:21.201040 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a85bb-b79b-498f-b6b1-983f01342097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:21.202603 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a85bb-b79b-498f-b6b1-983f01342097 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:21.213535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1352/2700] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:32:21.217440 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:21.226953 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed0c1a-5ee8-446d-aa06-1571e0b9c2e5 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:21.231468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:21.232365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:21.232648 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:21.232921 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] PATH_INFO: `/v3/projects/3c0d3f3daacc41ac8d7f458b7e3ff067/users/6603f7ed648e4d4b9d4ae55acb4fe1e3/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:21.247286 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0d3f3daacc41ac8d7f458b7e3ff067, user_id=6603f7ed648e4d4b9d4ae55acb4fe1e3, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:21.249944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:21.263886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1349/2701] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:32:21 2022] PUT /identity/v3/projects/3c0d3f3daacc41ac8d7f458b7e3ff067/users/6603f7ed648e4d4b9d4ae55acb4fe1e3/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:32:21.273997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8ce30-c17f-48c4-80dd-9c691a97d5ce None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:21.274284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8ce30-c17f-48c4-80dd-9c691a97d5ce None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:21.274565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8ce30-c17f-48c4-80dd-9c691a97d5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:21.301140 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4ff8ce30-c17f-48c4-80dd-9c691a97d5ce None None] MFA Rules not processed for user `6603f7ed648e4d4b9d4ae55acb4fe1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:21.302318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4ff8ce30-c17f-48c4-80dd-9c691a97d5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:21.367041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1353/2702] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:21.377664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:21.379441 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:21.400837 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:21.437631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f2190-92ed-44a7-80cd-10006437ebd1 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:21.441459 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:21.442091 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:21.442371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:21.442642 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:21.450803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:21.452200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:21.460756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1350/2703] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:32:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:22.791322 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3404309-d703-4d72-8560-52d681375db5 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:22.792007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3404309-d703-4d72-8560-52d681375db5 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:22.792007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3404309-d703-4d72-8560-52d681375db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:22.809061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f3404309-d703-4d72-8560-52d681375db5 None None] MFA Rules not processed for user `6603f7ed648e4d4b9d4ae55acb4fe1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:22.810260 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f3404309-d703-4d72-8560-52d681375db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:22.818216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1354/2704] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:23.725559 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:23.726944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:23.801803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb045c36-7267-42cf-8450-4b958167afe4 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:23.806673 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:23.807310 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:23.807592 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:23.807899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:23.809625 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:23.821271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:23.822908 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:23.832843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1351/2705] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:32:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:32.803800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:33.828576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=49 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:37.446733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdd95-4308-4fec-8e5d-d36980db1b81 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:37.447191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdd95-4308-4fec-8e5d-d36980db1b81 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:37.447309 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdd95-4308-4fec-8e5d-d36980db1b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:37.464921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-f6afdd95-4308-4fec-8e5d-d36980db1b81 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:37.466115 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-f6afdd95-4308-4fec-8e5d-d36980db1b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:37.520557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1355/2706] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:32:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:37.572088 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:37.573493 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:37.640604 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b7be-431d-4b4a-8d1c-ba33aef4ebfd service placement] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:37.645502 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:37.646134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:37.646418 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:37.646694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:37.654417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:37.656012 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:37.663745 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1352/2707] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:32:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:32:38.150824 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd9156-1f33-4c70-b2fd-67064b64fcff None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:38.151121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd9156-1f33-4c70-b2fd-67064b64fcff None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:38.151435 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd9156-1f33-4c70-b2fd-67064b64fcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:38.168860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-cccd9156-1f33-4c70-b2fd-67064b64fcff None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:38.170048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-cccd9156-1f33-4c70-b2fd-67064b64fcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:38.177363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1356/2708] 10.222.0.62 () {58 vars in 967 bytes} [Fri May 27 03:32:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:38.189749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:38.201082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5039a2-0d78-4fb6-9481-bea65f366423 service swift] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:38.206049 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:38.206670 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:38.206950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:38.207222 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:38.215111 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:38.216778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:38.224562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1353/2709] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:32:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:32:38.410709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b34c0d-7864-4a69-b3c6-2c050c3ac0b9 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:38.411001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b34c0d-7864-4a69-b3c6-2c050c3ac0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:38.411316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b34c0d-7864-4a69-b3c6-2c050c3ac0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:38.428457 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-e3b34c0d-7864-4a69-b3c6-2c050c3ac0b9 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:38.430116 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-e3b34c0d-7864-4a69-b3c6-2c050c3ac0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:38.437016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1357/2710] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:32:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:38.449241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:38.459501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5825c-d597-45b9-b7ec-ad809a91c6b5 service swift] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:38.472293 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:38.472930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:38.473212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:38.473490 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:38.482172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:38.483725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:38.493151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1354/2711] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:32:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:32:38.815314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148f65e-9c77-43c6-a238-8feb02f3e456 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:38.815781 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148f65e-9c77-43c6-a238-8feb02f3e456 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:38.815901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148f65e-9c77-43c6-a238-8feb02f3e456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:38.833222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d148f65e-9c77-43c6-a238-8feb02f3e456 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:38.834388 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d148f65e-9c77-43c6-a238-8feb02f3e456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:38.843759 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1358/2712] 10.222.0.62 () {58 vars in 967 bytes} [Fri May 27 03:32:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:38.856113 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:38.866400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89a557-b734-4162-a9e0-c4f4aa7c6542 service swift] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:38.871480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5496434-d2bf-4794-a3c2-33ea3026e03f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:38.872085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5496434-d2bf-4794-a3c2-33ea3026e03f service swift] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:38.872366 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5496434-d2bf-4794-a3c2-33ea3026e03f service swift] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:38.872639 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5496434-d2bf-4794-a3c2-33ea3026e03f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:38.884628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5496434-d2bf-4794-a3c2-33ea3026e03f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:38.886232 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5496434-d2bf-4794-a3c2-33ea3026e03f service swift] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:38.896630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1355/2713] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:32:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:32:40.914878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148f65e-9c77-43c6-a238-8feb02f3e456 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:40.976490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148f65e-9c77-43c6-a238-8feb02f3e456 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:40.980956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:40.981823 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:40.982108 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:40.982385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] PATH_INFO: `/v3/users/6603f7ed648e4d4b9d4ae55acb4fe1e3` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:40.983557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6603f7ed648e4d4b9d4ae55acb4fe1e3)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:40.985517 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.030932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6603f7ed648e4d4b9d4ae55acb4fe1e3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.031579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6603f7ed648e4d4b9d4ae55acb4fe1e3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.039625 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6603f7ed648e4d4b9d4ae55acb4fe1e3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.040311 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6603f7ed648e4d4b9d4ae55acb4fe1e3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.051673 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6603f7ed648e4d4b9d4ae55acb4fe1e3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.064138 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1359/2714] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:32:40 2022] DELETE /identity/v3/users/6603f7ed648e4d4b9d4ae55acb4fe1e3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:41.068452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.069205 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.069301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.086901 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:41.088157 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:41.145197 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1356/2715] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:41.154619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.156001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:41.162405 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:41.212840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a4986-162e-47f4-96f7-3a44b8d3d0bb admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.216414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d163f31-77c3-405d-90fb-d33bc59dde26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.217049 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d163f31-77c3-405d-90fb-d33bc59dde26 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.217330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d163f31-77c3-405d-90fb-d33bc59dde26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.217610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d163f31-77c3-405d-90fb-d33bc59dde26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.226379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d163f31-77c3-405d-90fb-d33bc59dde26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.227786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d163f31-77c3-405d-90fb-d33bc59dde26 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.237237 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1360/2716] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:41.398707 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.400211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:41.413838 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f05a32-ca66-471e-ae3c-228fb44ad4b3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.418979 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.420376 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.420664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.421106 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] PATH_INFO: `/v3/projects/3c0d3f3daacc41ac8d7f458b7e3ff067` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.422320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0d3f3daacc41ac8d7f458b7e3ff067)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.424325 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.468246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:41.471433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0d3f3daacc41ac8d7f458b7e3ff067', 'request_id': 'req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.479349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0d3f3daacc41ac8d7f458b7e3ff067', 'request_id': 'req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.484910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0d3f3daacc41ac8d7f458b7e3ff067', 'request_id': 'req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.496061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] The token cache is being invalidate because project 3c0d3f3daacc41ac8d7f458b7e3ff067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:41.496345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.496964 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-5e185c34-b8c0-40fe-96bb-0ab0aeb8326f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:41.498472 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1357/2717] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:32:41 2022] DELETE /identity/v3/projects/3c0d3f3daacc41ac8d7f458b7e3ff067 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:41.503363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366d36e-a610-4adb-9f7f-85a16a145136 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.503657 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366d36e-a610-4adb-9f7f-85a16a145136 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.503967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366d36e-a610-4adb-9f7f-85a16a145136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.521924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0366d36e-a610-4adb-9f7f-85a16a145136 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:41.523119 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0366d36e-a610-4adb-9f7f-85a16a145136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:41.581333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1361/2718] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:41.589009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48ddf4-c452-468c-9217-5e7e44ffca85 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.589296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48ddf4-c452-468c-9217-5e7e44ffca85 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.589572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48ddf4-c452-468c-9217-5e7e44ffca85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.608791 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-cb48ddf4-c452-468c-9217-5e7e44ffca85 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:41.609962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cb48ddf4-c452-468c-9217-5e7e44ffca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:41.617650 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1358/2719] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:41.622284 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366d36e-a610-4adb-9f7f-85a16a145136 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.631718 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366d36e-a610-4adb-9f7f-85a16a145136 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.636794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.637418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.637705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.637983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.638710 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:41.639021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.640649 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.652575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1362/2720] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:41.656903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48ddf4-c452-468c-9217-5e7e44ffca85 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.665930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48ddf4-c452-468c-9217-5e7e44ffca85 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.670511 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.671134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.671413 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.671717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.672442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.674359 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.699487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1359/2721] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:32:41 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:32:41.703453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.713751 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14a4e-ac49-48e5-9f2b-620351b8342a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.767677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.768403 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.768721 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.768994 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.769682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.771623 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.793948 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1363/2722] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:32:41 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:32:41.795378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.805872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aeb15-1a6b-46e1-989a-05e6571893e6 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.810984 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.811641 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.811972 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.812253 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.813006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.814578 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.825663 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1360/2723] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:32:41.829481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.839396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09a539-8408-4ed1-b970-cef69b6243cf admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.844191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8ad36-e990-4078-9d1a-400e8bff8ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.844852 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8ad36-e990-4078-9d1a-400e8bff8ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.845136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8ad36-e990-4078-9d1a-400e8bff8ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.845413 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8ad36-e990-4078-9d1a-400e8bff8ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.846147 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8ad36-e990-4078-9d1a-400e8bff8ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.847739 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8ad36-e990-4078-9d1a-400e8bff8ba1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.860478 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1364/2724] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:32:41.864728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:41.875054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43469878-14fb-443e-8e5f-adf06c3a0167 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:41.880029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:41.880904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.881191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.881468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] PATH_INFO: `/v3/projects/54f9fd67a8e2485eba03336c3b0b6b4e/users/5220c5f89d8b4841922a592cbd268c4b/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.896557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f9fd67a8e2485eba03336c3b0b6b4e, user_id=5220c5f89d8b4841922a592cbd268c4b, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:41.899204 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:41.911728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1361/2725] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:32:41 2022] PUT /identity/v3/projects/54f9fd67a8e2485eba03336c3b0b6b4e/users/5220c5f89d8b4841922a592cbd268c4b/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:41.916712 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c59e-7dd2-42a8-9514-e9d3928d9b83 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:41.917171 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c59e-7dd2-42a8-9514-e9d3928d9b83 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:41.917327 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c59e-7dd2-42a8-9514-e9d3928d9b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:41.936842 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3553c59e-7dd2-42a8-9514-e9d3928d9b83 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:41.938061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3553c59e-7dd2-42a8-9514-e9d3928d9b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:41.946630 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:41.998039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1365/2726] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:32:42.007529 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:42.009213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:42.065085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7805ced-004d-46a7-af4c-ce9847c731d5 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:42.068761 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:42.069668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:42.069960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:42.070242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:42.079834 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:42.081271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:42.090562 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1362/2727] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:46.300884 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459106-7703-4c24-9b88-db6ead611960 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:46.301465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459106-7703-4c24-9b88-db6ead611960 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:46.301465 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459106-7703-4c24-9b88-db6ead611960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:46.334328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-c3459106-7703-4c24-9b88-db6ead611960 None None] MFA Rules not processed for user `5220c5f89d8b4841922a592cbd268c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:46.335549 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-c3459106-7703-4c24-9b88-db6ead611960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:46.402959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1366/2728] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:46.407602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:46.418126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9237-a925-4a3f-8ae7-ac13387e266a service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:46.422705 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:46.423365 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:46.423651 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:46.423961 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:46.424669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:46.426741 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:46.453392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1363/2729] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:32:46 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:32:46.457595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3459106-7703-4c24-9b88-db6ead611960 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:46.469458 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3459106-7703-4c24-9b88-db6ead611960 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:46.528523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:46.529204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:46.529490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:46.529770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:46.530468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:46.532543 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:46.553255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1367/2730] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:32:46 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:32:46.557255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:46.567919 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8881e-5dfa-4d15-88a6-dd822061b78e admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:46.572831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:46.573518 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:46.573808 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:46.574087 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:46.574819 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:46.576487 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:46.588682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1364/2731] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:32:46.592605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:46.602933 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1935382-d770-4627-ab1f-963683f02cd9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:46.608705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:46.609635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:46.609929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:46.610211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] PATH_INFO: `/v3/projects/0dc01eb63c1440bc86fc0e01ee8e10e7/users/3ed6233bf6af48e4b4e0023a34fd05ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:46.625997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc01eb63c1440bc86fc0e01ee8e10e7, user_id=3ed6233bf6af48e4b4e0023a34fd05ec, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:46.628742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:46.641557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1368/2732] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:32:46 2022] PUT /identity/v3/projects/0dc01eb63c1440bc86fc0e01ee8e10e7/users/3ed6233bf6af48e4b4e0023a34fd05ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:32:46.645600 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:46.656260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad57af5-9dfa-467f-a0c8-0812608d657b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:46.694364 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:46.714950 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68752862-35ca-40ae-94b8-a100e94fb25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:46.715575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68752862-35ca-40ae-94b8-a100e94fb25b admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:46.715888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68752862-35ca-40ae-94b8-a100e94fb25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:46.716165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68752862-35ca-40ae-94b8-a100e94fb25b admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:46.716887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68752862-35ca-40ae-94b8-a100e94fb25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:46.718478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68752862-35ca-40ae-94b8-a100e94fb25b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:46.727965 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1365/2733] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:32:46.732100 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:46.742352 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c522-ab9b-4124-be1e-31ba3bc42fc9 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:46.747061 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:46.747976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:46.748267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:46.748547 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] PATH_INFO: `/v3/projects/0dc01eb63c1440bc86fc0e01ee8e10e7/users/3ed6233bf6af48e4b4e0023a34fd05ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:46.750521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc01eb63c1440bc86fc0e01ee8e10e7, user_id=3ed6233bf6af48e4b4e0023a34fd05ec, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:46.753191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:46.756176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:46.765341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1369/2734] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:32:46 2022] PUT /identity/v3/projects/0dc01eb63c1440bc86fc0e01ee8e10e7/users/3ed6233bf6af48e4b4e0023a34fd05ec/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:51.537941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c5ffd-1bdb-4fec-8c61-57085cd95d71 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:51.538431 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c5ffd-1bdb-4fec-8c61-57085cd95d71 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:51.538524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c5ffd-1bdb-4fec-8c61-57085cd95d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:51.613501 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-584c5ffd-1bdb-4fec-8c61-57085cd95d71 None None] MFA Rules not processed for user `3ed6233bf6af48e4b4e0023a34fd05ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:51.615125 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-584c5ffd-1bdb-4fec-8c61-57085cd95d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:51.801187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1366/2735] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:32:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:51.821873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:51.824528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:51.937813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddeb88-52eb-4065-8381-c9572f709d76 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:51.944566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7be3a546aa644281a0d3594395bbdf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:51.946285 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:51.947442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:51.948075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.039359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:52.041599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:52.060485 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1370/2736] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:32:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:52.225099 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57541033-7a01-4d00-9358-c7d21c747e4b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:52.225759 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57541033-7a01-4d00-9358-c7d21c747e4b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:52.226280 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57541033-7a01-4d00-9358-c7d21c747e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.248893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-57541033-7a01-4d00-9358-c7d21c747e4b None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:52.250428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-57541033-7a01-4d00-9358-c7d21c747e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:52.339936 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1367/2737] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:32:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:52.354083 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:52.357286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:52.462052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7e797-4237-402b-b0a4-a979ed114381 service glance] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:52.470054 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7655e-21f6-41a4-a5b1-a56459641461 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:52.470967 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7655e-21f6-41a4-a5b1-a56459641461 service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:52.471314 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7655e-21f6-41a4-a5b1-a56459641461 service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:52.471663 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7655e-21f6-41a4-a5b1-a56459641461 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.483169 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7655e-21f6-41a4-a5b1-a56459641461 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:52.484901 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7655e-21f6-41a4-a5b1-a56459641461 service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:52.527516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1371/2738] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:32:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:33:02.327494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=70 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:02.477805 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cd20a-f9a3-48a9-bd34-81e0887276e2 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:02.478496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cd20a-f9a3-48a9-bd34-81e0887276e2 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:02.478977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cd20a-f9a3-48a9-bd34-81e0887276e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:02.500575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=82 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:02.502401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-207cd20a-f9a3-48a9-bd34-81e0887276e2 None None] MFA Rules not processed for user `5220c5f89d8b4841922a592cbd268c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:02.503863 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-207cd20a-f9a3-48a9-bd34-81e0887276e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:02.515390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1368/2739] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:33:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:03.768983 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7588813-3c7b-400c-bfc5-cbde8c5c2b19 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:03.769461 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7588813-3c7b-400c-bfc5-cbde8c5c2b19 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:03.769572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7588813-3c7b-400c-bfc5-cbde8c5c2b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:03.786869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b7588813-3c7b-400c-bfc5-cbde8c5c2b19 None None] MFA Rules not processed for user `4bf0025854d64392810c6c9303eb9320`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:03.788277 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b7588813-3c7b-400c-bfc5-cbde8c5c2b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:03.853153 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1372/2740] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:33:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:05.226558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:05.227066 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:05.227184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:05.244840 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:05.246027 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:05.254057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1369/2741] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:33:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:05.269949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7588813-3c7b-400c-bfc5-cbde8c5c2b19 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:05.283762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7588813-3c7b-400c-bfc5-cbde8c5c2b19 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:05.289200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadee23-39b2-45d5-b19e-bbce446ae03a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:05.289966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadee23-39b2-45d5-b19e-bbce446ae03a service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:05.290256 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadee23-39b2-45d5-b19e-bbce446ae03a service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:05.291229 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadee23-39b2-45d5-b19e-bbce446ae03a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:05.301333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadee23-39b2-45d5-b19e-bbce446ae03a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:05.302927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadee23-39b2-45d5-b19e-bbce446ae03a service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:05.313550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1373/2742] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:33:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:33:15.239477 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:15.309310 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=31 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:28.810536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:28.812631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:28.875283 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e671-494b-40cb-bb96-4b54a3778a43 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:28.879749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:28.880427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:28.880717 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:28.881018 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:28.882803 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:28.942810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:28.944289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:28.953320 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1370/2743] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:33:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:32.199546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee171cd-884c-4825-9b3d-0c7bef3606e0 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:32.200131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee171cd-884c-4825-9b3d-0c7bef3606e0 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:32.200131 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee171cd-884c-4825-9b3d-0c7bef3606e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:32.218414 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7ee171cd-884c-4825-9b3d-0c7bef3606e0 None None] MFA Rules not processed for user `5c8ad338eb7b47c18dcb86f287e28858`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:32.219599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7ee171cd-884c-4825-9b3d-0c7bef3606e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:32.226758 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1374/2744] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:33:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:35.084085 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:35.205495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844f95c-9289-42d5-9a52-5064440c800e service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:35.209386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:35.210123 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:35.210484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:35.210955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:35.225400 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:35.226983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:35.239753 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1371/2745] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:33:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:42.213102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:45.233268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=74 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:01.468023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee171cd-884c-4825-9b3d-0c7bef3606e0 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:01.477944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee171cd-884c-4825-9b3d-0c7bef3606e0 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:01.481991 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:01.482635 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:01.482921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:01.483204 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:01.485037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:01.561436 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:01.562873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:01.573921 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1375/2746] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:34:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:02.754716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:02.756188 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:02.767354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710cfd-abc8-41c0-b2cc-62c9ff57e062 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:02.770918 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885da2ff-6b53-447a-b34e-4286d446f8ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:02.771547 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885da2ff-6b53-447a-b34e-4286d446f8ad service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:02.771859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885da2ff-6b53-447a-b34e-4286d446f8ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:02.772145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885da2ff-6b53-447a-b34e-4286d446f8ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:02.773853 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-885da2ff-6b53-447a-b34e-4286d446f8ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:02.786712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885da2ff-6b53-447a-b34e-4286d446f8ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:02.788191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885da2ff-6b53-447a-b34e-4286d446f8ad service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:02.799470 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1372/2747] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:34:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:11.568920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=34 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:12.794611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=6 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:22.975903 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:22.977724 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:22.992598 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd7f4-3135-46ba-9df8-ea8289e81445 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:23.011641 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:23.012399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:23.012687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:23.012971 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:23.015747 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:23.028908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:23.030613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:23.067245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1376/2748] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:34:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:25.954695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81777-3af5-49a2-8d13-78b45b20194f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:25.955191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81777-3af5-49a2-8d13-78b45b20194f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:25.955307 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81777-3af5-49a2-8d13-78b45b20194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:25.972201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-57e81777-3af5-49a2-8d13-78b45b20194f None None] MFA Rules not processed for user `4bf0025854d64392810c6c9303eb9320`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:34:25.973390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-57e81777-3af5-49a2-8d13-78b45b20194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:25.980191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1373/2749] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:34:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:34:26.582255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:26.583658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:26.655966 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49349-7b42-4a74-8658-45ec3d83a700 service nova] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:26.660924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9607b79-9408-4b94-8430-b9c0c5cd92f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:26.661568 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9607b79-9408-4b94-8430-b9c0c5cd92f6 service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:26.661949 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9607b79-9408-4b94-8430-b9c0c5cd92f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:26.662230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9607b79-9408-4b94-8430-b9c0c5cd92f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:26.664070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a9607b79-9408-4b94-8430-b9c0c5cd92f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:26.675682 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9607b79-9408-4b94-8430-b9c0c5cd92f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:26.677286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9607b79-9408-4b94-8430-b9c0c5cd92f6 service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:26.687453 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1377/2750] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:34:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:28.522166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81777-3af5-49a2-8d13-78b45b20194f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:28.533495 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81777-3af5-49a2-8d13-78b45b20194f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:28.550285 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:28.550958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:28.551241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:28.551516 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:28.553675 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:28.564982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:28.566550 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:28.575303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1374/2751] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:34:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:36.682799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=38 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:38.571430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=9 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:00.974359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4beb0-6af4-469d-8ae8-7901ffdbd6ce None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:00.974872 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4beb0-6af4-469d-8ae8-7901ffdbd6ce None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:00.974976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4beb0-6af4-469d-8ae8-7901ffdbd6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:00.992098 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-b3b4beb0-6af4-469d-8ae8-7901ffdbd6ce None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:00.993265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-b3b4beb0-6af4-469d-8ae8-7901ffdbd6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:01.000196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1378/2752] 10.222.0.62 () {58 vars in 967 bytes} [Fri May 27 03:35:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:01.040475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:01.052935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3df23-9265-41f0-95fe-082b6d0b3752 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:01.057859 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:01.058656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:01.058946 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:01.059225 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:01.067915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:01.069489 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:01.077391 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1375/2753] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:35:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:35:07.750704 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4beb0-6af4-469d-8ae8-7901ffdbd6ce None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.831491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4beb0-6af4-469d-8ae8-7901ffdbd6ce None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.836016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.836959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.837245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.837518 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] PATH_INFO: `/v3/users/5220c5f89d8b4841922a592cbd268c4b` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.838717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5220c5f89d8b4841922a592cbd268c4b)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.840916 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.888157 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5220c5f89d8b4841922a592cbd268c4b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:07.888810 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5220c5f89d8b4841922a592cbd268c4b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:07.899191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5220c5f89d8b4841922a592cbd268c4b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:07.899805 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5220c5f89d8b4841922a592cbd268c4b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:07.913222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5220c5f89d8b4841922a592cbd268c4b'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:07.925536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1379/2754] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:07 2022] DELETE /identity/v3/users/5220c5f89d8b4841922a592cbd268c4b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:08.100184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.110596 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c1850-8e25-4446-a3e9-34d935e5d1cf service swift] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.167514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.168408 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.168695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.168973 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] PATH_INFO: `/v3/projects/54f9fd67a8e2485eba03336c3b0b6b4e` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.170147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f9fd67a8e2485eba03336c3b0b6b4e)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.172075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.215880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f9fd67a8e2485eba03336c3b0b6b4e', 'request_id': 'req-cffa231b-5999-4b0f-b77d-c7d9808c2f84'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.222896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f9fd67a8e2485eba03336c3b0b6b4e', 'request_id': 'req-cffa231b-5999-4b0f-b77d-c7d9808c2f84'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.228975 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f9fd67a8e2485eba03336c3b0b6b4e', 'request_id': 'req-cffa231b-5999-4b0f-b77d-c7d9808c2f84'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.240990 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] The token cache is being invalidate because project 54f9fd67a8e2485eba03336c3b0b6b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:08.241276 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.241897 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.243674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1376/2755] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:08 2022] DELETE /identity/v3/projects/54f9fd67a8e2485eba03336c3b0b6b4e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:08.247370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.248779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:08.261348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ccb0-c4ed-477c-b4ef-7c9cb78dddfb admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.316170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.317063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.317342 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.317615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] PATH_INFO: `/v3/users/3ed6233bf6af48e4b4e0023a34fd05ec` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.318777 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed6233bf6af48e4b4e0023a34fd05ec)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.320717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.334022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:08.367757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed6233bf6af48e4b4e0023a34fd05ec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.368361 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed6233bf6af48e4b4e0023a34fd05ec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.375488 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed6233bf6af48e4b4e0023a34fd05ec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.376321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed6233bf6af48e4b4e0023a34fd05ec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.388192 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed6233bf6af48e4b4e0023a34fd05ec'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.400576 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1380/2756] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:08 2022] DELETE /identity/v3/users/3ed6233bf6af48e4b4e0023a34fd05ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:08.568794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.570291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:08.584882 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa231b-5999-4b0f-b77d-c7d9808c2f84 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.649064 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.649849 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.650134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.650411 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] PATH_INFO: `/v3/projects/0dc01eb63c1440bc86fc0e01ee8e10e7` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.651631 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc01eb63c1440bc86fc0e01ee8e10e7)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.653605 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.675881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:08.702139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc01eb63c1440bc86fc0e01ee8e10e7', 'request_id': 'req-1b67a94c-895a-4647-a025-2bd685c2b39a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.709581 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc01eb63c1440bc86fc0e01ee8e10e7', 'request_id': 'req-1b67a94c-895a-4647-a025-2bd685c2b39a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.715397 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc01eb63c1440bc86fc0e01ee8e10e7', 'request_id': 'req-1b67a94c-895a-4647-a025-2bd685c2b39a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.726135 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] The token cache is being invalidate because project 0dc01eb63c1440bc86fc0e01ee8e10e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:08.726422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.727075 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:08.728684 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1377/2757] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:08 2022] DELETE /identity/v3/projects/0dc01eb63c1440bc86fc0e01ee8e10e7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:18.397070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:18.723029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:36.072163 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.073752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:36.088031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640b7d-5692-4bc7-b16b-953a7eee20b4 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.154172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.155212 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.155504 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.155786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] PATH_INFO: `/v3/users/4bf0025854d64392810c6c9303eb9320` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.159120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf0025854d64392810c6c9303eb9320)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.159120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.207389 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf0025854d64392810c6c9303eb9320'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.208005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf0025854d64392810c6c9303eb9320'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.215380 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf0025854d64392810c6c9303eb9320'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.216015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf0025854d64392810c6c9303eb9320'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.228456 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6abc8f4e-1a3a-4a5e-b856-509e25a5b2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf0025854d64392810c6c9303eb9320'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.240329 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1381/2758] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:36 2022] DELETE /identity/v3/users/4bf0025854d64392810c6c9303eb9320 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:36.410695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.412189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:36.427117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a94c-895a-4647-a025-2bd685c2b39a admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.485659 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.486527 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.486815 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.487092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] PATH_INFO: `/v3/projects/6a6c40fbca064c789708300a00c36ce4` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.488322 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6c40fbca064c789708300a00c36ce4)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.490260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.536336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6c40fbca064c789708300a00c36ce4', 'request_id': 'req-e4335c90-c2b8-409f-ac33-f527abb45465'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.543312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6c40fbca064c789708300a00c36ce4', 'request_id': 'req-e4335c90-c2b8-409f-ac33-f527abb45465'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.548774 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6c40fbca064c789708300a00c36ce4', 'request_id': 'req-e4335c90-c2b8-409f-ac33-f527abb45465'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.559330 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] The token cache is being invalidate because project 6a6c40fbca064c789708300a00c36ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:36.559612 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.560257 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-e4335c90-c2b8-409f-ac33-f527abb45465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:36.561874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1378/2759] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:36 2022] DELETE /identity/v3/projects/6a6c40fbca064c789708300a00c36ce4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:36.566746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a243ede-3fc4-4459-a3d6-5a226dc964ef None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.567044 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a243ede-3fc4-4459-a3d6-5a226dc964ef None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.567375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a243ede-3fc4-4459-a3d6-5a226dc964ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.587421 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-2a243ede-3fc4-4459-a3d6-5a226dc964ef None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:36.588661 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-2a243ede-3fc4-4459-a3d6-5a226dc964ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:36.645146 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1382/2760] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:36.653396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fa79a-c54d-4f11-b50f-74d2ff8bc9de None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.653689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fa79a-c54d-4f11-b50f-74d2ff8bc9de None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.653969 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fa79a-c54d-4f11-b50f-74d2ff8bc9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.672461 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-485fa79a-c54d-4f11-b50f-74d2ff8bc9de None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:36.673655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-485fa79a-c54d-4f11-b50f-74d2ff8bc9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:36.682162 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1379/2761] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:36.686084 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a243ede-3fc4-4459-a3d6-5a226dc964ef None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.695509 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a243ede-3fc4-4459-a3d6-5a226dc964ef None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.699986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.700637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.700927 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.701203 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.701930 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:36.702238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.703839 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.715063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1383/2762] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:36.719395 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fa79a-c54d-4f11-b50f-74d2ff8bc9de None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.728907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fa79a-c54d-4f11-b50f-74d2ff8bc9de None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.733768 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.734424 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.734710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.734991 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.735689 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.737658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.763116 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1380/2763] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:35:36 2022] POST /identity/v3/projects => generated 405 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:36.770330 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.780723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcca0b1-f3ee-4c2f-95ee-310379eb0017 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.784182 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:36.837017 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.837672 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.837955 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.838233 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.838919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.840846 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.861523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1384/2764] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:35:36 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:36.865823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.876482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76117270-c52f-4b64-a353-3f7d99a3e896 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.881061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.881767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.882054 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.882335 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.883092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.884731 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.894907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1381/2765] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:36.899030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.908732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f5db-6b67-4204-a405-cbb9c7bb971c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.914022 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712eac84-9a63-4d4b-bbbe-cc9e98ad9138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.914679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712eac84-9a63-4d4b-bbbe-cc9e98ad9138 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.914972 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712eac84-9a63-4d4b-bbbe-cc9e98ad9138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.915385 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712eac84-9a63-4d4b-bbbe-cc9e98ad9138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.916052 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712eac84-9a63-4d4b-bbbe-cc9e98ad9138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.917669 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712eac84-9a63-4d4b-bbbe-cc9e98ad9138 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.928431 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1385/2766] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:36.932616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.943804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0e7f-6ffd-4b9b-bb2e-d437c02ce129 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.948845 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.949729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.950039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.950316 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] PATH_INFO: `/v3/projects/1a82d93d41654f5799cecccab30142be/users/f1287366328e43319f6320b74db31eaa/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.967118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a82d93d41654f5799cecccab30142be, user_id=f1287366328e43319f6320b74db31eaa, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.969931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.983903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1382/2767] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:36 2022] PUT /identity/v3/projects/1a82d93d41654f5799cecccab30142be/users/f1287366328e43319f6320b74db31eaa/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:36.994963 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280caf-46fb-426b-a93b-7061543c2c4f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.995251 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280caf-46fb-426b-a93b-7061543c2c4f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.995542 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280caf-46fb-426b-a93b-7061543c2c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.022735 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-88280caf-46fb-426b-a93b-7061543c2c4f None None] MFA Rules not processed for user `f1287366328e43319f6320b74db31eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:37.024018 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-88280caf-46fb-426b-a93b-7061543c2c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:37.086524 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1386/2768] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:37.090931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.120826 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:37.154615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a2e23-d0de-4bf9-9a76-e96d13cc32c6 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.159250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.159929 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.160216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.160494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.161181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.163620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.189193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1383/2769] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:35:37 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:37.193179 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88280caf-46fb-426b-a93b-7061543c2c4f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.203503 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88280caf-46fb-426b-a93b-7061543c2c4f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.258208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.258860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.259145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.259420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.260135 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.262087 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.281418 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1387/2770] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:35:37 2022] POST /identity/v3/users => generated 309 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:37.285903 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.297668 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fca9d-3cf6-4774-aa85-0aa737cdd972 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.302272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.302926 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.303218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.304375 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.305171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.306827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.318742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1384/2771] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:37.323132 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.329513 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:37.333789 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b071ff-4d3f-4b01-b429-57fd0c02290d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.339587 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.340762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.341053 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.341371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] PATH_INFO: `/v3/projects/b926192e78fa4464b0bfb06df5596e4b/users/f693a82bd03f429fa9ce350985e1cb9d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.358976 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b926192e78fa4464b0bfb06df5596e4b, user_id=f693a82bd03f429fa9ce350985e1cb9d, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.361866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.375493 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1388/2772] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:37 2022] PUT /identity/v3/projects/b926192e78fa4464b0bfb06df5596e4b/users/f693a82bd03f429fa9ce350985e1cb9d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:37.379564 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.389590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8a075-7d6b-406d-b73b-ff229c2913ce admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.445218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dab12-a73c-4d4d-842d-17adb3f6de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.445888 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dab12-a73c-4d4d-842d-17adb3f6de29 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.446152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dab12-a73c-4d4d-842d-17adb3f6de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.446430 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dab12-a73c-4d4d-842d-17adb3f6de29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.447164 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dab12-a73c-4d4d-842d-17adb3f6de29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.448807 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dab12-a73c-4d4d-842d-17adb3f6de29 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.458301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1385/2773] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:35:37.462550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.472993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602887-092f-41a9-b08d-6da0332ebe3e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.478101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.479170 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.479452 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.479743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] PATH_INFO: `/v3/projects/b926192e78fa4464b0bfb06df5596e4b/users/f693a82bd03f429fa9ce350985e1cb9d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.481779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b926192e78fa4464b0bfb06df5596e4b, user_id=f693a82bd03f429fa9ce350985e1cb9d, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.484442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.495162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1389/2774] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:37 2022] PUT /identity/v3/projects/b926192e78fa4464b0bfb06df5596e4b/users/f693a82bd03f429fa9ce350985e1cb9d/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:37.505156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4b23e-6880-4425-885b-42af6e8d53e8 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.505447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4b23e-6880-4425-885b-42af6e8d53e8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.505728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4b23e-6880-4425-885b-42af6e8d53e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.532385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-cbf4b23e-6880-4425-885b-42af6e8d53e8 None None] MFA Rules not processed for user `f693a82bd03f429fa9ce350985e1cb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:37.533517 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cbf4b23e-6880-4425-885b-42af6e8d53e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:37.600268 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1386/2775] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:37.611334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.613060 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:37.668868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd797-facd-496e-8ae2-07e8a9860a8e admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.672535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240eb2d5-105e-4dac-8d58-fd1c68e618dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.673176 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240eb2d5-105e-4dac-8d58-fd1c68e618dc service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.673463 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240eb2d5-105e-4dac-8d58-fd1c68e618dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.673746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240eb2d5-105e-4dac-8d58-fd1c68e618dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.682795 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240eb2d5-105e-4dac-8d58-fd1c68e618dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.684261 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240eb2d5-105e-4dac-8d58-fd1c68e618dc service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.694349 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1390/2776] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:37 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:37.834438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4b23e-6880-4425-885b-42af6e8d53e8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.862120 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:37.900756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4b23e-6880-4425-885b-42af6e8d53e8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.905288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.906185 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.906469 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.906751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] PATH_INFO: `/v3/users/f1287366328e43319f6320b74db31eaa` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.907969 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1287366328e43319f6320b74db31eaa)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.909891 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:37.956989 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1287366328e43319f6320b74db31eaa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:37.957647 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1287366328e43319f6320b74db31eaa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:37.958247 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1287366328e43319f6320b74db31eaa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:37.971037 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1287366328e43319f6320b74db31eaa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:37.978591 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1287366328e43319f6320b74db31eaa'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:37.991008 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1387/2777] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:37 2022] DELETE /identity/v3/users/f1287366328e43319f6320b74db31eaa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:37.995572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.995956 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.996241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:38.014161 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:38.015360 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:38.079848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1391/2778] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:38.090457 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:38.092166 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:38.152291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294ba0-db8e-458a-89d6-86bd529dc79b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:38.156076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:38.156725 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:38.157004 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:38.157277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:38.166212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:38.167635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:38.177151 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1388/2779] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:38.445150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:38.446602 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:38.458914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c383e5d-279b-4098-9bb3-5b1e3b04e5e4 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:38.466961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:38.468742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:38.469081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:38.469501 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] PATH_INFO: `/v3/projects/1a82d93d41654f5799cecccab30142be` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:38.470754 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a82d93d41654f5799cecccab30142be)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:38.472889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:38.524448 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a82d93d41654f5799cecccab30142be', 'request_id': 'req-de13c4df-1b1d-40de-9c00-999e9b60ff7f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.530508 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a82d93d41654f5799cecccab30142be', 'request_id': 'req-de13c4df-1b1d-40de-9c00-999e9b60ff7f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.535952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a82d93d41654f5799cecccab30142be', 'request_id': 'req-de13c4df-1b1d-40de-9c00-999e9b60ff7f'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.538992 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:38.552484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] The token cache is being invalidate because project 1a82d93d41654f5799cecccab30142be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:38.553021 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.553650 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.555510 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1392/2780] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:38 2022] DELETE /identity/v3/projects/1a82d93d41654f5799cecccab30142be => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:38.559385 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:38.569870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b27c-9bf2-4e16-8592-c4a3058c6e65 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:38.613463 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:38.631209 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04664034-4235-4188-935f-76cce2117742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:38.632674 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664034-4235-4188-935f-76cce2117742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:38.632958 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664034-4235-4188-935f-76cce2117742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:38.633235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664034-4235-4188-935f-76cce2117742 admin admin] PATH_INFO: `/v3/users/f693a82bd03f429fa9ce350985e1cb9d` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:38.634645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664034-4235-4188-935f-76cce2117742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f693a82bd03f429fa9ce350985e1cb9d)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:38.636611 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664034-4235-4188-935f-76cce2117742 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:38.682781 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-04664034-4235-4188-935f-76cce2117742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f693a82bd03f429fa9ce350985e1cb9d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.683436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-04664034-4235-4188-935f-76cce2117742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f693a82bd03f429fa9ce350985e1cb9d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.684062 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-04664034-4235-4188-935f-76cce2117742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f693a82bd03f429fa9ce350985e1cb9d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.696554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-04664034-4235-4188-935f-76cce2117742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f693a82bd03f429fa9ce350985e1cb9d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.703606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-04664034-4235-4188-935f-76cce2117742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f693a82bd03f429fa9ce350985e1cb9d'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.718874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1389/2781] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:38 2022] DELETE /identity/v3/users/f693a82bd03f429fa9ce350985e1cb9d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:39.093569 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.095013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.109743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c4df-1b1d-40de-9c00-999e9b60ff7f admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.180982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.181909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.182208 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.182490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] PATH_INFO: `/v3/projects/b926192e78fa4464b0bfb06df5596e4b` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.183702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b926192e78fa4464b0bfb06df5596e4b)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.185687 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.241114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b926192e78fa4464b0bfb06df5596e4b', 'request_id': 'req-eab71d94-27e7-4950-b820-cca199728288'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.247201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b926192e78fa4464b0bfb06df5596e4b', 'request_id': 'req-eab71d94-27e7-4950-b820-cca199728288'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.252671 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b926192e78fa4464b0bfb06df5596e4b', 'request_id': 'req-eab71d94-27e7-4950-b820-cca199728288'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.265606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] The token cache is being invalidate because project b926192e78fa4464b0bfb06df5596e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:39.265800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.266467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-eab71d94-27e7-4950-b820-cca199728288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.268045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1393/2782] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:39 2022] DELETE /identity/v3/projects/b926192e78fa4464b0bfb06df5596e4b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:39.273532 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932ad4d-d418-4ab6-9aac-eb26e58efc70 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.273895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932ad4d-d418-4ab6-9aac-eb26e58efc70 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.274179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932ad4d-d418-4ab6-9aac-eb26e58efc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.291952 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e932ad4d-d418-4ab6-9aac-eb26e58efc70 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:39.293144 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e932ad4d-d418-4ab6-9aac-eb26e58efc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.352565 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1390/2783] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:39.360334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c557b-90cd-43ae-bfc6-23fbdbaf4a3c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.361096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c557b-90cd-43ae-bfc6-23fbdbaf4a3c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.361536 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c557b-90cd-43ae-bfc6-23fbdbaf4a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.379840 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-560c557b-90cd-43ae-bfc6-23fbdbaf4a3c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:39.381074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-560c557b-90cd-43ae-bfc6-23fbdbaf4a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.389371 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1394/2784] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:39.393704 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932ad4d-d418-4ab6-9aac-eb26e58efc70 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.405158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932ad4d-d418-4ab6-9aac-eb26e58efc70 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.410245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.410910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.411194 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.411475 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.412241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:39.412552 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.414153 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.425827 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1391/2785] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:39.430356 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c557b-90cd-43ae-bfc6-23fbdbaf4a3c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.439440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c557b-90cd-43ae-bfc6-23fbdbaf4a3c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.444634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.445294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.445869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.445869 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.446546 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.448499 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.477149 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1395/2786] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/projects => generated 357 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:39.478525 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.488496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc414e9-79da-4c33-ab78-f172f2eb2c57 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.536383 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:39.548326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.549005 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.549289 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.549572 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.550288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.552246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.575614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1392/2787] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:39.577563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.588847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae88dc-d41a-44d5-aa90-78342d3f3298 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.593950 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.594658 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.594947 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.595227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.595999 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.597617 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.608962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1396/2788] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:39.612721 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.623267 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162ea89-592a-4581-b339-f3e3e9db6b10 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.628092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fff2b0-6a3d-4115-a584-06f9c0af551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.628767 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fff2b0-6a3d-4115-a584-06f9c0af551c admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.629057 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fff2b0-6a3d-4115-a584-06f9c0af551c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.629345 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fff2b0-6a3d-4115-a584-06f9c0af551c admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.630118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fff2b0-6a3d-4115-a584-06f9c0af551c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.632281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fff2b0-6a3d-4115-a584-06f9c0af551c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.642913 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1393/2789] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:39.647064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.657861 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7be54-4b12-4628-b255-96f599933993 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.662534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.664045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.664334 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.664613 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] PATH_INFO: `/v3/projects/da2273f7e12f4d7aa0824f2185fd5294/users/a00036c6c5284f78b870aa19939db14c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.680215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da2273f7e12f4d7aa0824f2185fd5294, user_id=a00036c6c5284f78b870aa19939db14c, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.683030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.697875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1397/2790] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:39 2022] PUT /identity/v3/projects/da2273f7e12f4d7aa0824f2185fd5294/users/a00036c6c5284f78b870aa19939db14c/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:39.709221 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14633367-54c5-4454-92fc-0ff8fdf0efa8 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.709513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14633367-54c5-4454-92fc-0ff8fdf0efa8 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.709830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14633367-54c5-4454-92fc-0ff8fdf0efa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.738473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-14633367-54c5-4454-92fc-0ff8fdf0efa8 None None] MFA Rules not processed for user `a00036c6c5284f78b870aa19939db14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:39.739669 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-14633367-54c5-4454-92fc-0ff8fdf0efa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.809343 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1394/2791] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:39.813941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.834470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:39.880875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87735abb-aa46-4f4d-a53a-f7222e836115 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.885717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.886378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.886779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.887070 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.887782 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.889740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.916283 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1398/2792] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:39.920427 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14633367-54c5-4454-92fc-0ff8fdf0efa8 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.931514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14633367-54c5-4454-92fc-0ff8fdf0efa8 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.988577 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.989243 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.989524 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.989800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.990492 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.992468 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:40.011599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1395/2793] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:40.015656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:40.026213 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de25f41-25b0-48d9-bf26-7eeae2b7b58b admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:40.030771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:40.031424 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:40.031709 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:40.032023 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:40.032757 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:40.034382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:40.045317 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1399/2794] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:40.049265 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:40.059114 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470df371-ffe8-4741-a33b-e3c52b5fa9ce admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:40.063728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:40.064638 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:40.064927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:40.065211 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] PATH_INFO: `/v3/projects/38ec8925ca8046f79e7e5212208d289c/users/d0c105d22a504ec482a9167ccf13edc2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:40.079665 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ec8925ca8046f79e7e5212208d289c, user_id=d0c105d22a504ec482a9167ccf13edc2, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:40.082479 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:40.095432 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1396/2795] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:40 2022] PUT /identity/v3/projects/38ec8925ca8046f79e7e5212208d289c/users/d0c105d22a504ec482a9167ccf13edc2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:40.099368 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:40.109957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ec1bc-087a-4e82-8d41-494249edf2e5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:40.166550 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966ce0-aa83-46d4-84ff-699b3d998035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:40.167125 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966ce0-aa83-46d4-84ff-699b3d998035 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:40.167408 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966ce0-aa83-46d4-84ff-699b3d998035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:40.167686 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966ce0-aa83-46d4-84ff-699b3d998035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:40.168447 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966ce0-aa83-46d4-84ff-699b3d998035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:40.170034 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966ce0-aa83-46d4-84ff-699b3d998035 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:40.180101 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1400/2796] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:40.184856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:40.195880 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d37ce-dcad-43b9-936d-192110eee92f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:40.201246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:40.202128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:40.202407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:40.202677 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] PATH_INFO: `/v3/projects/38ec8925ca8046f79e7e5212208d289c/users/d0c105d22a504ec482a9167ccf13edc2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:40.204655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ec8925ca8046f79e7e5212208d289c, user_id=d0c105d22a504ec482a9167ccf13edc2, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:40.207273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:40.218927 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1397/2797] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:40 2022] PUT /identity/v3/projects/38ec8925ca8046f79e7e5212208d289c/users/d0c105d22a504ec482a9167ccf13edc2/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:40.228741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7a44e-a4f2-4a74-bf58-2f765ec83d8f None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:40.229030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7a44e-a4f2-4a74-bf58-2f765ec83d8f None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:40.229306 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7a44e-a4f2-4a74-bf58-2f765ec83d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:40.255151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-3ac7a44e-a4f2-4a74-bf58-2f765ec83d8f None None] MFA Rules not processed for user `d0c105d22a504ec482a9167ccf13edc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:40.256392 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3ac7a44e-a4f2-4a74-bf58-2f765ec83d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:40.326198 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1401/2798] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:40 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:40.338428 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:40.340406 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:40.353925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:40.399935 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272691-d01d-4a6b-bbde-a467f7f0dea2 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:40.404234 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:40.404886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:40.405171 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:40.405447 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:40.414687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:40.416278 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:40.425787 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1398/2799] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:40 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:40.498717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7a44e-a4f2-4a74-bf58-2f765ec83d8f None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:40.508811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7a44e-a4f2-4a74-bf58-2f765ec83d8f None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:40.512373 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:40.513008 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:40.513295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:40.513572 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:40.525907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:40.572628 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:40.574000 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:40.583121 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1402/2800] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:40 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:44.677069 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:44.678453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.738076 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1484881-1184-491b-82f3-510921027a9c service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:44.742386 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:44.743093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.743403 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.743764 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:44.745618 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.814246 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:44.815694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:44.826729 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1399/2801] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:47.627719 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.630051 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:47.749173 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdacec9-858e-42d0-be5e-379c397de732 service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.759595 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.760440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.761397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.761788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.764341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:47.782975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.785067 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.796868 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1403/2802] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:50.923233 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.924854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:50.944865 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ade8d-027a-4f22-ac74-6259ab5808af service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.950671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.951255 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.951542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.951842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] PATH_INFO: `/v3/users/5c8ad338eb7b47c18dcb86f287e28858` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.953109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8ad338eb7b47c18dcb86f287e28858)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.955329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:51.003422 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8ad338eb7b47c18dcb86f287e28858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.004102 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8ad338eb7b47c18dcb86f287e28858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.004715 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8ad338eb7b47c18dcb86f287e28858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.012970 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:51.019412 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8ad338eb7b47c18dcb86f287e28858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.026371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8ad338eb7b47c18dcb86f287e28858'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.039488 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1400/2803] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:50 2022] DELETE /identity/v3/users/5c8ad338eb7b47c18dcb86f287e28858 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:51.207450 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:51.218183 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072068-a0f4-4217-bc3c-ad084237ed7a service placement] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:51.274058 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:51.274914 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:51.275253 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:51.275535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] PATH_INFO: `/v3/projects/ffc47aa302dc4c4e94c4bc7227728c40` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:51.276748 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc47aa302dc4c4e94c4bc7227728c40)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:51.278770 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:51.328440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:51.331468 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc47aa302dc4c4e94c4bc7227728c40', 'request_id': 'req-4b527623-05ea-480c-a43c-0ebe0a9530fa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.338336 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc47aa302dc4c4e94c4bc7227728c40', 'request_id': 'req-4b527623-05ea-480c-a43c-0ebe0a9530fa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.345107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc47aa302dc4c4e94c4bc7227728c40', 'request_id': 'req-4b527623-05ea-480c-a43c-0ebe0a9530fa'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.357363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] The token cache is being invalidate because project ffc47aa302dc4c4e94c4bc7227728c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:51.357651 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.358266 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.359792 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1404/2804] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:51 2022] DELETE /identity/v3/projects/ffc47aa302dc4c4e94c4bc7227728c40 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:51.363992 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:51.365396 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:51.382121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095755a-e44f-4155-96ca-12aa2ca54439 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:51.443602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:51.444599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:51.444886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:51.445165 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] PATH_INFO: `/v3/users/5558f4ffc4c947978b34643f09b20d32` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:51.446509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5558f4ffc4c947978b34643f09b20d32)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:51.448548 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:51.494718 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5558f4ffc4c947978b34643f09b20d32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.495356 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5558f4ffc4c947978b34643f09b20d32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.495995 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5558f4ffc4c947978b34643f09b20d32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.507582 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5558f4ffc4c947978b34643f09b20d32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.514329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5558f4ffc4c947978b34643f09b20d32'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.526702 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1401/2805] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:51 2022] DELETE /identity/v3/users/5558f4ffc4c947978b34643f09b20d32 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:51.693664 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:51.695150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:51.709455 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527623-05ea-480c-a43c-0ebe0a9530fa admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:51.769622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:51.770491 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:51.770780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:51.771064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] PATH_INFO: `/v3/projects/73b3a58043fe4fdf92acdca722e152f5` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:51.772289 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73b3a58043fe4fdf92acdca722e152f5)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:51.774853 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:51.822926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73b3a58043fe4fdf92acdca722e152f5', 'request_id': 'req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.829136 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73b3a58043fe4fdf92acdca722e152f5', 'request_id': 'req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.835043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73b3a58043fe4fdf92acdca722e152f5', 'request_id': 'req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.849565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] The token cache is being invalidate because project 73b3a58043fe4fdf92acdca722e152f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:51.849855 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.850481 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-8cac25f7-e1f2-49de-bd4a-c2e9e03fdb47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.852832 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1405/2806] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:51 2022] DELETE /identity/v3/projects/73b3a58043fe4fdf92acdca722e152f5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:55.387138 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:55.388643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:55.401301 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c52ef-67c4-443d-adc9-bd7ae3c10783 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:55.450124 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:55.457941 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:55.458986 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:55.459484 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:55.459812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] PATH_INFO: `/v3/users/a00036c6c5284f78b870aa19939db14c` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:55.461189 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00036c6c5284f78b870aa19939db14c)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:55.463139 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:55.508656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00036c6c5284f78b870aa19939db14c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:55.509288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00036c6c5284f78b870aa19939db14c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:55.509881 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00036c6c5284f78b870aa19939db14c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:55.521587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00036c6c5284f78b870aa19939db14c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:55.528906 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00036c6c5284f78b870aa19939db14c'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:55.541655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1402/2807] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:55 2022] DELETE /identity/v3/users/a00036c6c5284f78b870aa19939db14c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:55.546252 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:55.546771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:55.546886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:55.564088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:55.565275 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:55.622575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1406/2808] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:55.632122 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:55.633551 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:55.689756 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf593a-159f-4899-90a9-4d89bbd07dd4 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:55.693452 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:55.694092 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:55.694379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:55.694658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:55.703736 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:55.705142 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:55.714887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1403/2809] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:55.979268 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:55.980732 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:55.989159 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:55.992803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024aae3-6e10-42cc-93c8-8d1f28747bdd None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:55.998998 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:56.000565 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.000908 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.001200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] PATH_INFO: `/v3/projects/da2273f7e12f4d7aa0824f2185fd5294` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.002717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da2273f7e12f4d7aa0824f2185fd5294)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:56.005001 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:56.052894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da2273f7e12f4d7aa0824f2185fd5294', 'request_id': 'req-3424a7eb-bca2-4467-a0ab-868ab05a42c1'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.058923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da2273f7e12f4d7aa0824f2185fd5294', 'request_id': 'req-3424a7eb-bca2-4467-a0ab-868ab05a42c1'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.064456 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da2273f7e12f4d7aa0824f2185fd5294', 'request_id': 'req-3424a7eb-bca2-4467-a0ab-868ab05a42c1'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.076800 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] The token cache is being invalidate because project da2273f7e12f4d7aa0824f2185fd5294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:56.077086 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.077702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.079240 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1407/2810] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:55 2022] DELETE /identity/v3/projects/da2273f7e12f4d7aa0824f2185fd5294 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:56.083262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:56.093730 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d660bfb-8a73-48f5-9deb-28131993c00f service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:56.148473 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:56.149948 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.150240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.150515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] PATH_INFO: `/v3/users/d0c105d22a504ec482a9167ccf13edc2` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.151711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c105d22a504ec482a9167ccf13edc2)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:56.153719 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:56.173443 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:56.199621 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c105d22a504ec482a9167ccf13edc2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.200291 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c105d22a504ec482a9167ccf13edc2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.200898 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c105d22a504ec482a9167ccf13edc2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.212960 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c105d22a504ec482a9167ccf13edc2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.220158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-da9572e2-8c6f-49fb-a9cd-82daf44acfb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c105d22a504ec482a9167ccf13edc2'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.232568 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1404/2811] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:35:56 2022] DELETE /identity/v3/users/d0c105d22a504ec482a9167ccf13edc2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:56.500483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:56.501926 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:56.514594 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424a7eb-bca2-4467-a0ab-868ab05a42c1 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:56.572961 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:56.573829 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.574118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.574398 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] PATH_INFO: `/v3/projects/38ec8925ca8046f79e7e5212208d289c` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.575570 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ec8925ca8046f79e7e5212208d289c)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:56.577523 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:56.623232 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ec8925ca8046f79e7e5212208d289c', 'request_id': 'req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.629433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ec8925ca8046f79e7e5212208d289c', 'request_id': 'req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.635312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ec8925ca8046f79e7e5212208d289c', 'request_id': 'req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.647013 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] The token cache is being invalidate because project 38ec8925ca8046f79e7e5212208d289c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:56.647300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.647957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ee4d85f0-638e-4f38-a93c-9a4100cc4fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:56.650088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1408/2812] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:35:56 2022] DELETE /identity/v3/projects/38ec8925ca8046f79e7e5212208d289c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:56.654944 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a412d7-8853-4e5c-b509-bbc7e34eab0b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.655455 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a412d7-8853-4e5c-b509-bbc7e34eab0b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.655570 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a412d7-8853-4e5c-b509-bbc7e34eab0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.673117 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-b5a412d7-8853-4e5c-b509-bbc7e34eab0b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:56.674295 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-b5a412d7-8853-4e5c-b509-bbc7e34eab0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:56.733390 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1405/2813] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:56.741154 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73667491-9664-433d-92e3-3f3611455bee None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.741470 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73667491-9664-433d-92e3-3f3611455bee None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.741750 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73667491-9664-433d-92e3-3f3611455bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.759970 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-73667491-9664-433d-92e3-3f3611455bee None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:56.761077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-73667491-9664-433d-92e3-3f3611455bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:56.768741 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1409/2814] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:56.772646 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a412d7-8853-4e5c-b509-bbc7e34eab0b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:56.783078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a412d7-8853-4e5c-b509-bbc7e34eab0b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:56.787645 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:56.788344 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.788628 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.788907 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.789643 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:56.790245 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:56.792218 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:56.804830 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1406/2815] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:56.809242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73667491-9664-433d-92e3-3f3611455bee None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:56.818575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73667491-9664-433d-92e3-3f3611455bee None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:56.823987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:56.824645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.824930 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.825211 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.825920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:56.827871 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:56.835975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:56.859571 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1410/2816] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:35:56 2022] POST /identity/v3/projects => generated 395 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:56.860829 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:56.870726 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cbba2-fcc9-4c8d-9980-b701831f56f0 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:56.933227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:56.933895 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.934178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.934453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.935158 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:56.937212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:56.957288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1407/2817] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:35:56 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:56.961401 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:56.972442 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6a7dc-a3e9-4d06-96da-d3c35270e347 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:56.983746 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:56.984422 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:56.984706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:56.984982 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:56.985702 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:56.987280 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:56.997500 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1411/2818] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:57.001349 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.016174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:57.055836 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03ba29-ecaf-4b19-9e91-51a0cdaab79f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.060558 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef7c28-19eb-4c93-8f5a-3cf576dbc3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.061288 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef7c28-19eb-4c93-8f5a-3cf576dbc3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.061575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef7c28-19eb-4c93-8f5a-3cf576dbc3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.061855 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef7c28-19eb-4c93-8f5a-3cf576dbc3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.062584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef7c28-19eb-4c93-8f5a-3cf576dbc3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.064576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef7c28-19eb-4c93-8f5a-3cf576dbc3f5 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.075201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1408/2819] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:57.079300 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.115110 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54905ae6-c0b0-43f2-9083-cf536e7450b7 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.120209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.121088 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.121374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.121653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] PATH_INFO: `/v3/projects/49c13b1d490f4de39038bebbfaf43bb4/users/6f1e1f59df1a4cbcbcf28cb8917ab653/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.135700 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c13b1d490f4de39038bebbfaf43bb4, user_id=6f1e1f59df1a4cbcbcf28cb8917ab653, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.138379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.162987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1412/2820] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:57 2022] PUT /identity/v3/projects/49c13b1d490f4de39038bebbfaf43bb4/users/6f1e1f59df1a4cbcbcf28cb8917ab653/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:57.173951 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf75a2c-a09e-484c-b844-117eda5057bb None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.174263 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf75a2c-a09e-484c-b844-117eda5057bb None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.174537 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf75a2c-a09e-484c-b844-117eda5057bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.209931 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-fcf75a2c-a09e-484c-b844-117eda5057bb None None] MFA Rules not processed for user `6f1e1f59df1a4cbcbcf28cb8917ab653`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:57.211126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-fcf75a2c-a09e-484c-b844-117eda5057bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:57.272179 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1409/2821] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:57.276788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.337997 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bba86-c61e-4606-8882-09cbecf75453 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.342728 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.343642 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.343957 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.344242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.344932 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.346856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.371653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1413/2822] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:35:57 2022] POST /identity/v3/projects => generated 401 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:57.375602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf75a2c-a09e-484c-b844-117eda5057bb None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.386509 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf75a2c-a09e-484c-b844-117eda5057bb None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.440856 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.441554 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.441842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.442128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.442821 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.444981 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.464811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1410/2823] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:35:57 2022] POST /identity/v3/users => generated 307 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:57.472646 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.483446 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26422f26-e5e7-451a-8c47-f0c23d5d49be admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.493665 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.494328 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.494607 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.494878 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.495610 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.497222 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.507875 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1414/2824] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:57.512213 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.522392 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cd612-e4ef-4d56-8f70-a209803e5fb3 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.526999 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.527910 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.528198 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.528505 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] PATH_INFO: `/v3/projects/3403006323de465bb08e879855803821/users/bfd281ebe32c46b2bd031dd589dde38b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.543332 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3403006323de465bb08e879855803821, user_id=bfd281ebe32c46b2bd031dd589dde38b, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.546029 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.559590 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1411/2825] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:57 2022] PUT /identity/v3/projects/3403006323de465bb08e879855803821/users/bfd281ebe32c46b2bd031dd589dde38b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:57.563502 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.573374 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5e64c-c64f-492c-8764-fbf43f7f6ebd admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.614615 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:57.630490 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c12a7a-a579-408b-8899-525bd19558c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.631150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12a7a-a579-408b-8899-525bd19558c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.631441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12a7a-a579-408b-8899-525bd19558c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.631722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12a7a-a579-408b-8899-525bd19558c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.632497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12a7a-a579-408b-8899-525bd19558c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.634096 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12a7a-a579-408b-8899-525bd19558c6 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.643668 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1415/2826] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:57.647716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.658212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b48a2-4c4c-4f1b-a463-0609ce6b59ef admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.662846 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.663746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.664082 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.664361 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] PATH_INFO: `/v3/projects/3403006323de465bb08e879855803821/users/bfd281ebe32c46b2bd031dd589dde38b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.666810 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3403006323de465bb08e879855803821, user_id=bfd281ebe32c46b2bd031dd589dde38b, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.669496 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.672626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:57.682078 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1412/2827] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:35:57 2022] PUT /identity/v3/projects/3403006323de465bb08e879855803821/users/bfd281ebe32c46b2bd031dd589dde38b/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:57.692048 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070ddce-1a59-4ae5-890d-775fb8affa17 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.692346 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070ddce-1a59-4ae5-890d-775fb8affa17 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.692634 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070ddce-1a59-4ae5-890d-775fb8affa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.718799 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-0070ddce-1a59-4ae5-890d-775fb8affa17 None None] MFA Rules not processed for user `bfd281ebe32c46b2bd031dd589dde38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:57.720030 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-0070ddce-1a59-4ae5-890d-775fb8affa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:57.795397 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1416/2828] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:35:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:57.805956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.807616 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:57.880734 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae692f-abe9-4a7f-b0ca-756d2f558afb admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.884380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.885015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.885299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:57.885576 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:57.894304 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:57.895740 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:57.905389 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1413/2829] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:57.985267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070ddce-1a59-4ae5-890d-775fb8affa17 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:57.994897 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070ddce-1a59-4ae5-890d-775fb8affa17 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:57.998938 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a7a2e-0e21-4cd3-b664-c0987858f0a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:57.999582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7a2e-0e21-4cd3-b664-c0987858f0a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:57.999905 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7a2e-0e21-4cd3-b664-c0987858f0a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:58.000193 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7a2e-0e21-4cd3-b664-c0987858f0a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:58.055797 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a7a2e-0e21-4cd3-b664-c0987858f0a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:58.057318 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a7a2e-0e21-4cd3-b664-c0987858f0a1 service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:58.071655 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1417/2830] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:35:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:02.067794 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:02.128573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1936-9a66-4689-9a7f-c1965e2a0406 service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:02.133015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:02.133872 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:02.134152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:02.134467 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] PATH_INFO: `/v3/users/6f1e1f59df1a4cbcbcf28cb8917ab653` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:02.135627 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1e1f59df1a4cbcbcf28cb8917ab653)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:02.137603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:02.182465 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1e1f59df1a4cbcbcf28cb8917ab653'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.183109 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1e1f59df1a4cbcbcf28cb8917ab653'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.183716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1e1f59df1a4cbcbcf28cb8917ab653'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.195302 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1e1f59df1a4cbcbcf28cb8917ab653'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.202215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1e1f59df1a4cbcbcf28cb8917ab653'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.213679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1414/2831] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:36:02 2022] DELETE /identity/v3/users/6f1e1f59df1a4cbcbcf28cb8917ab653 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:02.218214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:02.218929 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:02.219027 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:02.245482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:02.246608 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:02.294778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=100 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:02.304064 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1418/2832] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:02.313569 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:02.315003 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:02.318925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:02.395712 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c1d12-6512-4b88-be65-3efa9ad32689 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:02.399371 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:02.400557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:02.400843 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:02.401115 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:02.409854 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:02.411281 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:02.421026 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1415/2833] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:02.686432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:02.688430 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:02.700831 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaac058-5dfb-442d-8c37-b0314a59db7e None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:02.710339 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:02.711216 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:02.711506 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:02.711786 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] PATH_INFO: `/v3/projects/49c13b1d490f4de39038bebbfaf43bb4` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:02.713575 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c13b1d490f4de39038bebbfaf43bb4)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:02.715534 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:02.761885 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c13b1d490f4de39038bebbfaf43bb4', 'request_id': 'req-96a48b3c-fc34-4f26-b17e-c48b3af960a8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.767899 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c13b1d490f4de39038bebbfaf43bb4', 'request_id': 'req-96a48b3c-fc34-4f26-b17e-c48b3af960a8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.773296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c13b1d490f4de39038bebbfaf43bb4', 'request_id': 'req-96a48b3c-fc34-4f26-b17e-c48b3af960a8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.785528 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] The token cache is being invalidate because project 49c13b1d490f4de39038bebbfaf43bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:02.785818 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.786441 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.787993 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1419/2834] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:36:02 2022] DELETE /identity/v3/projects/49c13b1d490f4de39038bebbfaf43bb4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:02.792160 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:02.801893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e0ec6-71d3-4b32-8e09-57a8c3ac9649 service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:02.864061 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:02.864947 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:02.865212 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:02.865486 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] PATH_INFO: `/v3/users/bfd281ebe32c46b2bd031dd589dde38b` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:02.866656 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd281ebe32c46b2bd031dd589dde38b)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:02.868615 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:02.913940 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd281ebe32c46b2bd031dd589dde38b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.914584 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd281ebe32c46b2bd031dd589dde38b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.915191 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd281ebe32c46b2bd031dd589dde38b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.927256 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd281ebe32c46b2bd031dd589dde38b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.934235 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-c1e139db-189f-49fa-b52a-6b9bc1764b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd281ebe32c46b2bd031dd589dde38b'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:02.946250 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1416/2835] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:36:02 2022] DELETE /identity/v3/users/bfd281ebe32c46b2bd031dd589dde38b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:03.221535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:03.223059 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:03.236409 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a48b3c-fc34-4f26-b17e-c48b3af960a8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:03.292241 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:03.293037 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.293326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.293611 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] PATH_INFO: `/v3/projects/3403006323de465bb08e879855803821` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.294790 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3403006323de465bb08e879855803821)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:03.296749 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:03.343812 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3403006323de465bb08e879855803821', 'request_id': 'req-6d89ca08-5b41-4d1f-9f03-8767705a86ae'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:03.349694 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3403006323de465bb08e879855803821', 'request_id': 'req-6d89ca08-5b41-4d1f-9f03-8767705a86ae'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:03.355743 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3403006323de465bb08e879855803821', 'request_id': 'req-6d89ca08-5b41-4d1f-9f03-8767705a86ae'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:03.367514 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] The token cache is being invalidate because project 3403006323de465bb08e879855803821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:03.367808 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:03.368531 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-6d89ca08-5b41-4d1f-9f03-8767705a86ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:03.370633 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1420/2836] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:36:03 2022] DELETE /identity/v3/projects/3403006323de465bb08e879855803821 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:03.375575 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcecfd-b0f4-4b63-8546-8bee9a0604d1 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.376086 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcecfd-b0f4-4b63-8546-8bee9a0604d1 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.376195 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcecfd-b0f4-4b63-8546-8bee9a0604d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.381121 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:03.414678 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-f4bcecfd-b0f4-4b63-8546-8bee9a0604d1 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:03.416407 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-f4bcecfd-b0f4-4b63-8546-8bee9a0604d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:03.475119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1417/2837] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:03.486155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e825b25-2add-4db1-aa25-1b0275c7cb80 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.486155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e825b25-2add-4db1-aa25-1b0275c7cb80 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.486155 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e825b25-2add-4db1-aa25-1b0275c7cb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.504697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-9e825b25-2add-4db1-aa25-1b0275c7cb80 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:03.506378 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9e825b25-2add-4db1-aa25-1b0275c7cb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:03.515771 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1421/2838] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:03.520060 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcecfd-b0f4-4b63-8546-8bee9a0604d1 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:03.529700 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcecfd-b0f4-4b63-8546-8bee9a0604d1 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:03.534482 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:03.535126 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.535405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.535676 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.536442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:03.536751 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:03.538336 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:03.550187 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1418/2839] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:03.554321 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e825b25-2add-4db1-aa25-1b0275c7cb80 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:03.560423 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:03.564364 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e825b25-2add-4db1-aa25-1b0275c7cb80 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:03.570255 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:03.571038 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.571494 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.571959 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.572722 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:03.574766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:03.599394 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1422/2840] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:36:03 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:03.604252 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:03.613842 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab311e56-e595-4dd0-9196-c1ed365d860f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:03.668379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:03.669039 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.669326 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.669606 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.670297 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:03.672272 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:03.692028 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1419/2841] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:36:03 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:03.696036 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:03.706475 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4020dc-57a0-4f3e-bff8-27e6db1457aa admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:03.711118 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:03.711778 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.712102 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.712387 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.713145 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:03.714744 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:03.727043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1423/2842] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:03.731438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:03.741242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeca6e-17fb-48ca-8f94-943ac9a17147 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:03.746312 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f933c1c-b1e8-464c-94f1-315156fdad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:03.746957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f933c1c-b1e8-464c-94f1-315156fdad96 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.747244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f933c1c-b1e8-464c-94f1-315156fdad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.747521 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f933c1c-b1e8-464c-94f1-315156fdad96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.748271 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f933c1c-b1e8-464c-94f1-315156fdad96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:03.749848 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f933c1c-b1e8-464c-94f1-315156fdad96 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:03.759728 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1420/2843] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:03.763714 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:03.773637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c424c29-fa1c-444a-9d24-28898fc60ed1 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:03.778774 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:03.779652 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.780004 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.780286 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] PATH_INFO: `/v3/projects/c4bbad5b907c44d98e0dab2c80b4d3c8/users/91c8ebced3034e0b8672d8d22578ee9f/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.795267 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4bbad5b907c44d98e0dab2c80b4d3c8, user_id=91c8ebced3034e0b8672d8d22578ee9f, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:03.807892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:03.831562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1424/2844] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:36:03 2022] PUT /identity/v3/projects/c4bbad5b907c44d98e0dab2c80b4d3c8/users/91c8ebced3034e0b8672d8d22578ee9f/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:03.842401 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55877f-53d5-4126-9e43-fc38023f8aab None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:03.842694 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55877f-53d5-4126-9e43-fc38023f8aab None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:03.842976 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55877f-53d5-4126-9e43-fc38023f8aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:03.870861 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-cc55877f-53d5-4126-9e43-fc38023f8aab None None] MFA Rules not processed for user `91c8ebced3034e0b8672d8d22578ee9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:03.872377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-cc55877f-53d5-4126-9e43-fc38023f8aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:03.916789 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:03.938436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1421/2845] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:03.942849 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.004521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ac60-d9ae-42f3-95e2-78b695b99f0a admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.009226 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.010599 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.010891 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.011174 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.011909 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.013813 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.038047 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1425/2846] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:36:03 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:04.042287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55877f-53d5-4126-9e43-fc38023f8aab None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.052654 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55877f-53d5-4126-9e43-fc38023f8aab None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.113655 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.114329 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.114613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.114890 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.115589 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.117557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.138377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1422/2847] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:36:04 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:04.141776 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.151946 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058901d8-13e3-4b97-a164-e37c25b1dbc3 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.156522 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.157246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.157535 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.157817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.158579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.160224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.171606 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1426/2848] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:04.175448 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.185957 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d8fc-6b86-460c-a379-c8bd7528afc6 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.190599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.191474 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.191762 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.192093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] PATH_INFO: `/v3/projects/aa3aa7c512e74b4299eabc8f23d971a2/users/a19245af604345568410cb86d87d960a/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.206339 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3aa7c512e74b4299eabc8f23d971a2, user_id=a19245af604345568410cb86d87d960a, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.209021 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.221679 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1423/2849] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:36:04 2022] PUT /identity/v3/projects/aa3aa7c512e74b4299eabc8f23d971a2/users/a19245af604345568410cb86d87d960a/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:04.225784 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.236186 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7accf7-7c39-4e13-a8f7-b0d7e9dfaa92 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.291893 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12be7f-655c-46f8-9ca1-d3b4dfa9f261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.292563 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12be7f-655c-46f8-9ca1-d3b4dfa9f261 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.292854 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12be7f-655c-46f8-9ca1-d3b4dfa9f261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.293151 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12be7f-655c-46f8-9ca1-d3b4dfa9f261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.293873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12be7f-655c-46f8-9ca1-d3b4dfa9f261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.295473 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12be7f-655c-46f8-9ca1-d3b4dfa9f261 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.305095 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1427/2850] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:04.310007 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.320186 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c194104-c56f-4452-82ed-166d1c804965 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.324766 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.325637 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.325922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.326216 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] PATH_INFO: `/v3/projects/aa3aa7c512e74b4299eabc8f23d971a2/users/a19245af604345568410cb86d87d960a/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.328203 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3aa7c512e74b4299eabc8f23d971a2, user_id=a19245af604345568410cb86d87d960a, role_id=e79af7a2ff0c4ea4ad04fd2ea4f5f25f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.330862 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.340955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1424/2851] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:36:04 2022] PUT /identity/v3/projects/aa3aa7c512e74b4299eabc8f23d971a2/users/a19245af604345568410cb86d87d960a/roles/e79af7a2ff0c4ea4ad04fd2ea4f5f25f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:04.351272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cacfa-9d8b-4997-a60a-6b70e7718c44 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.351562 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cacfa-9d8b-4997-a60a-6b70e7718c44 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.351866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cacfa-9d8b-4997-a60a-6b70e7718c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.378200 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-002cacfa-9d8b-4997-a60a-6b70e7718c44 None None] MFA Rules not processed for user `a19245af604345568410cb86d87d960a`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:04.379391 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-002cacfa-9d8b-4997-a60a-6b70e7718c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:04.384242 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:04.446498 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1428/2852] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:04.457974 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.459701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:04.515145 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2193c41-25f7-4f37-b86e-dfa49108a39f admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.518749 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.519370 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.519652 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.519961 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.529193 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.530603 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.539682 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1425/2853] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:04.610867 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cacfa-9d8b-4997-a60a-6b70e7718c44 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.620706 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cacfa-9d8b-4997-a60a-6b70e7718c44 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:04.624295 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd634615-e0b8-4e6d-9f2e-4a18c8b9c89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:04.624924 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd634615-e0b8-4e6d-9f2e-4a18c8b9c89b service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:04.625209 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd634615-e0b8-4e6d-9f2e-4a18c8b9c89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:04.625483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd634615-e0b8-4e6d-9f2e-4a18c8b9c89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:04.680558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd634615-e0b8-4e6d-9f2e-4a18c8b9c89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:04.682016 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd634615-e0b8-4e6d-9f2e-4a18c8b9c89b service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:04.690609 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1429/2854] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:04.941377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:04.984118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:05.004587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f5711-51d8-4ee1-a3f8-58313cbefb4d service cinder] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:05.009226 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:05.010649 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:05.010930 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:05.011202 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] PATH_INFO: `/v3/users/91c8ebced3034e0b8672d8d22578ee9f` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:05.012510 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c8ebced3034e0b8672d8d22578ee9f)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:05.014602 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:05.061595 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c8ebced3034e0b8672d8d22578ee9f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.062227 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c8ebced3034e0b8672d8d22578ee9f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.062915 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c8ebced3034e0b8672d8d22578ee9f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.074710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c8ebced3034e0b8672d8d22578ee9f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.081478 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c8ebced3034e0b8672d8d22578ee9f'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.093977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1426/2855] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:36:04 2022] DELETE /identity/v3/users/91c8ebced3034e0b8672d8d22578ee9f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:05.098592 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:05.098888 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:05.099172 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:05.116557 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:05.117681 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:05.189762 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1430/2856] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:05.199437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:05.200823 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:05.256379 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebec217-83cc-4b1a-854e-4972b8c4016c admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:05.260002 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:05.260635 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:05.260925 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:05.261201 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:05.270011 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:05.271436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:05.281219 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1427/2857] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:05.547516 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:05.548974 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:05.562767 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ec53f-d5a5-4cb8-81bc-77828cbf5572 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:05.567351 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:05.568316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:05.568583 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:05.569348 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] PATH_INFO: `/v3/projects/c4bbad5b907c44d98e0dab2c80b4d3c8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:05.570730 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4bbad5b907c44d98e0dab2c80b4d3c8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:05.573305 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:05.611196 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:05.621944 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4bbad5b907c44d98e0dab2c80b4d3c8', 'request_id': 'req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.628002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4bbad5b907c44d98e0dab2c80b4d3c8', 'request_id': 'req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.633375 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4bbad5b907c44d98e0dab2c80b4d3c8', 'request_id': 'req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.645335 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] The token cache is being invalidate because project c4bbad5b907c44d98e0dab2c80b4d3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:05.645622 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.646236 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.647742 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1431/2858] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:36:05 2022] DELETE /identity/v3/projects/c4bbad5b907c44d98e0dab2c80b4d3c8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:05.651886 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:05.661378 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09195-9491-49bb-b64d-5d2f4fd03dcf service neutron] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:05.716417 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:05.717346 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:05.717630 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:05.717912 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] PATH_INFO: `/v3/users/a19245af604345568410cb86d87d960a` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:05.719380 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19245af604345568410cb86d87d960a)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:05.721449 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:05.723688 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:05.768899 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19245af604345568410cb86d87d960a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.769531 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19245af604345568410cb86d87d960a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.770132 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19245af604345568410cb86d87d960a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.781716 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19245af604345568410cb86d87d960a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.788405 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-7cb1ca11-2ff4-450b-9239-55f818fd4791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19245af604345568410cb86d87d960a'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:05.801607 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1428/2859] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:36:05 2022] DELETE /identity/v3/users/a19245af604345568410cb86d87d960a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:06.073077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.074590 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:06.087359 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde1aba-5665-49ee-b5e5-fe05dbebaeb8 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.143600 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.144507 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.144794 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.145074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] PATH_INFO: `/v3/projects/aa3aa7c512e74b4299eabc8f23d971a2` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.146316 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3aa7c512e74b4299eabc8f23d971a2)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.148224 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.194428 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3aa7c512e74b4299eabc8f23d971a2', 'request_id': 'req-9b170101-fd68-4404-8274-ec64931dcba3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:06.201272 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3aa7c512e74b4299eabc8f23d971a2', 'request_id': 'req-9b170101-fd68-4404-8274-ec64931dcba3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:06.206798 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3aa7c512e74b4299eabc8f23d971a2', 'request_id': 'req-9b170101-fd68-4404-8274-ec64931dcba3'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:06.219107 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] The token cache is being invalidate because project aa3aa7c512e74b4299eabc8f23d971a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80273) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:06.219396 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:06.220045 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9b170101-fd68-4404-8274-ec64931dcba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:06.221579 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1432/2860] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:36:06 2022] DELETE /identity/v3/projects/aa3aa7c512e74b4299eabc8f23d971a2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:06.226557 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de436745-7aac-452e-a6c4-a3db5011c6a3 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.226852 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de436745-7aac-452e-a6c4-a3db5011c6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.227137 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de436745-7aac-452e-a6c4-a3db5011c6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.244954 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-de436745-7aac-452e-a6c4-a3db5011c6a3 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:06.246149 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-de436745-7aac-452e-a6c4-a3db5011c6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:06.304535 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1429/2861] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:06.366952 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d506f-8646-4d33-9354-2bbc2c29dbbd None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.367271 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d506f-8646-4d33-9354-2bbc2c29dbbd None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.367551 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d506f-8646-4d33-9354-2bbc2c29dbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.387451 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-719d506f-8646-4d33-9354-2bbc2c29dbbd None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:06.388919 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-719d506f-8646-4d33-9354-2bbc2c29dbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:06.397582 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1433/2862] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:06.401695 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de436745-7aac-452e-a6c4-a3db5011c6a3 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.411710 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de436745-7aac-452e-a6c4-a3db5011c6a3 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.416922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.417573 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.417857 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.418134 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] PATH_INFO: `/v3/domains` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.418866 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80272) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:06.419180 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.420825 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.432778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1430/2863] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:06.437304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d506f-8646-4d33-9354-2bbc2c29dbbd None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.446677 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d506f-8646-4d33-9354-2bbc2c29dbbd None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.451406 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.452091 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.452382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.452660 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] PATH_INFO: `/v3/projects` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.453363 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.455347 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.480484 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1434/2864] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:36:06 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:06.484613 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.494800 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1c4c-1884-412a-8520-405d078abfca admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.549098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.549754 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.550041 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.550317 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] PATH_INFO: `/v3/users` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.551011 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.552982 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.566614 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:06.572893 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1431/2865] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:36:06 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:06.576987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.587028 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde820-eaa6-42a1-9e47-9b6e5b6ba31c admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.591605 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.592308 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.592612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.592887 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.593637 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.595472 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.606697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1435/2866] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:06.610687 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.621287 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526c80d-e431-45fd-be49-37e1a4e7207d admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.625939 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7427c-3bd8-489b-98d0-0919ca791cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.626587 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7427c-3bd8-489b-98d0-0919ca791cba admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.626874 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7427c-3bd8-489b-98d0-0919ca791cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.627152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7427c-3bd8-489b-98d0-0919ca791cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.627914 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7427c-3bd8-489b-98d0-0919ca791cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.629508 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7427c-3bd8-489b-98d0-0919ca791cba admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.639127 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1432/2867] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:06.643214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.653215 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b328f5-ebd0-41ca-9a04-bc3503ea3670 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.657803 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.658679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.658965 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.659246 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] PATH_INFO: `/v3/projects/207f7211aa65416b8458950bc93ef06a/users/cb3739622b9d4e40a1efa8b286e01846/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.671986 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:06.675290 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=207f7211aa65416b8458950bc93ef06a, user_id=cb3739622b9d4e40a1efa8b286e01846, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.678527 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.692238 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1436/2868] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:36:06 2022] PUT /identity/v3/projects/207f7211aa65416b8458950bc93ef06a/users/cb3739622b9d4e40a1efa8b286e01846/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:06.697264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0df85b-3732-4382-ab5c-d004466218b0 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.697742 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0df85b-3732-4382-ab5c-d004466218b0 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.697844 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0df85b-3732-4382-ab5c-d004466218b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.715251 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-3e0df85b-3732-4382-ab5c-d004466218b0 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:06.716454 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-3e0df85b-3732-4382-ab5c-d004466218b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:06.773583 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1433/2869] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:06.783653 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:06.785341 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:06.840645 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25f5c1-7b8b-4b04-aead-3c74f49d1632 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:06.844497 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:06.845133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:06.845420 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:06.845703 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:06.855039 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:06.856619 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:06.865902 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1437/2870] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:07.474875 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727b420-1b0a-4784-8947-8e7ed1181580 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:07.475174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727b420-1b0a-4784-8947-8e7ed1181580 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:07.475464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727b420-1b0a-4784-8947-8e7ed1181580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:07.501671 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-4727b420-1b0a-4784-8947-8e7ed1181580 None None] MFA Rules not processed for user `cb3739622b9d4e40a1efa8b286e01846`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:07.502804 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-4727b420-1b0a-4784-8947-8e7ed1181580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:07.570206 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1434/2871] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:07.581379 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:07.582856 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:07.642007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf643b6f-2192-420d-b346-8c3cc182b2e9 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:07.645675 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:07.646326 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:07.646612 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:07.646892 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:07.656063 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:07.657566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:07.666294 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1438/2872] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:09.009410 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:09.009920 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:09.010019 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:09.028098 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] MFA Rules not processed for user `cb3739622b9d4e40a1efa8b286e01846`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:09.029279 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:09.036006 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1435/2873] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:09.266834 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:09.276281 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c47cd-14ad-4b3c-a53f-363e9754266f service cinder] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:09.279779 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a948d-d496-4a0e-9681-da50634645bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:09.280444 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a948d-d496-4a0e-9681-da50634645bf service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:09.280729 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a948d-d496-4a0e-9681-da50634645bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:09.281005 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a948d-d496-4a0e-9681-da50634645bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:09.282785 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-283a948d-d496-4a0e-9681-da50634645bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:09.292544 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a948d-d496-4a0e-9681-da50634645bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:09.293954 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a948d-d496-4a0e-9681-da50634645bf service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:09.301866 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1439/2874] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:16.795093 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:16.796785 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:16.875771 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7970-3c43-48ae-b42a-9dc53226209e None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:16.881231 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f5943-2373-45ea-806a-b129672b7569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc938bed535461e974daf01bb99e9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:16.881870 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5943-2373-45ea-806a-b129672b7569 service nova] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:16.882152 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5943-2373-45ea-806a-b129672b7569 service nova] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:16.882433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5943-2373-45ea-806a-b129672b7569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:16.884249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-165f5943-2373-45ea-806a-b129672b7569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:16.894148 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=100 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:16.896708 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f5943-2373-45ea-806a-b129672b7569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:16.898796 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f5943-2373-45ea-806a-b129672b7569 service nova] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:16.915354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1436/2875] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:19.298265 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:21.898245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:21.898864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:21.898864 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:21.916312 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] MFA Rules not processed for user `cb3739622b9d4e40a1efa8b286e01846`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:21.917522 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:21.925723 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1440/2876] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:26.905217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:31.911382 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:39.193701 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86876176-9c5b-43ba-a98d-8d2624e3b300 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:39.194354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86876176-9c5b-43ba-a98d-8d2624e3b300 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:39.194354 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86876176-9c5b-43ba-a98d-8d2624e3b300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:39.222896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-86876176-9c5b-43ba-a98d-8d2624e3b300 None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:39.224128 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-86876176-9c5b-43ba-a98d-8d2624e3b300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:39.284732 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1437/2877] 10.222.0.62 () {58 vars in 1009 bytes} [Fri May 27 03:36:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:39.329566 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:39.330987 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:39.399142 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69132a2a-5383-49c5-84b1-5cc0f21763b3 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:39.405033 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:39.405679 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:39.405964 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:39.406244 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:39.414345 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:39.415975 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:39.423900 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1441/2878] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:36:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:36:49.278068 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=32 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:49.420693 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=32 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:53.928434 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:53.928902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:53.929009 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:53.946296 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] MFA Rules not processed for user `1c908d4a33a041d8acc6ecad8c39ac6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:53.947480 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:53.954526 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1438/2879] 10.222.0.62 () {58 vars in 967 bytes} [Fri May 27 03:36:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:53.967486 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:53.978894 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679de1b-95e8-411e-910d-e8709ea59299 service swift] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:53.983807 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b633648662a4b51bad083fd4637f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:53.984467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:53.984752 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:53.985062 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:53.992923 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:53.994526 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:54.002440 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1442/2880] 10.222.0.62 () {60 vars in 1384 bytes} [Fri May 27 03:36:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:36:55.825746 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:55.827200 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:55.877185 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:55.888002 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde95db-9174-46c8-944c-8d1a3eccc49a service swift] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:55.893304 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:55.894227 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:55.894511 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:55.894920 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] PATH_INFO: `/v3/users/cb3739622b9d4e40a1efa8b286e01846` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:55.896150 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3739622b9d4e40a1efa8b286e01846)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:55.898075 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:55.905437 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8871d-1fb8-4f39-90db-7604d3e0d09f None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:55.910523 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:55.911172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:55.911456 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:55.911735 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:55.913536 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:55.926262 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:55.927994 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:55.942442 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1439/2881] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:55.947438 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3739622b9d4e40a1efa8b286e01846'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:55.948103 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3739622b9d4e40a1efa8b286e01846'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:55.957201 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3739622b9d4e40a1efa8b286e01846'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:55.957806 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3739622b9d4e40a1efa8b286e01846'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:55.971377 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-261233a0-a31a-427a-bf57-8a560d3c1547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3739622b9d4e40a1efa8b286e01846'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:55.984133 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1443/2882] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:36:55 2022] DELETE /identity/v3/users/cb3739622b9d4e40a1efa8b286e01846 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:56.155922 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.169795 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87850210-7354-428a-87e4-7c7d57c4b4c8 service placement] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.226277 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.227133 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.227415 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.227692 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] PATH_INFO: `/v3/projects/207f7211aa65416b8458950bc93ef06a` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.228896 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=207f7211aa65416b8458950bc93ef06a)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.230811 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.275174 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207f7211aa65416b8458950bc93ef06a', 'request_id': 'req-ee159f7c-8add-42f4-abcf-49794b695fbe'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:56.282015 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207f7211aa65416b8458950bc93ef06a', 'request_id': 'req-ee159f7c-8add-42f4-abcf-49794b695fbe'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:56.287382 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207f7211aa65416b8458950bc93ef06a', 'request_id': 'req-ee159f7c-8add-42f4-abcf-49794b695fbe'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:56.297831 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] The token cache is being invalidate because project 207f7211aa65416b8458950bc93ef06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:56.298119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:56.299242 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-ee159f7c-8add-42f4-abcf-49794b695fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:56.300778 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1440/2883] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:36:56 2022] DELETE /identity/v3/projects/207f7211aa65416b8458950bc93ef06a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:56.305720 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205c36d-fe59-4d18-a990-ceb3cc953f6c None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.306007 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205c36d-fe59-4d18-a990-ceb3cc953f6c None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.306282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205c36d-fe59-4d18-a990-ceb3cc953f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.334152 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-7205c36d-fe59-4d18-a990-ceb3cc953f6c None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:56.335333 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-7205c36d-fe59-4d18-a990-ceb3cc953f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:56.392081 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1444/2884] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:56.399887 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329219a4-64a1-4119-835c-03f5ab8f392b None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.400181 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329219a4-64a1-4119-835c-03f5ab8f392b None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.400464 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329219a4-64a1-4119-835c-03f5ab8f392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.418438 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-329219a4-64a1-4119-835c-03f5ab8f392b None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:56.419620 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-329219a4-64a1-4119-835c-03f5ab8f392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:56.427980 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1441/2885] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:56.431962 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205c36d-fe59-4d18-a990-ceb3cc953f6c None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.441788 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205c36d-fe59-4d18-a990-ceb3cc953f6c None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.446257 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.446907 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.447190 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.447464 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.448230 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80273) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:56.448540 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.450120 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.461296 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1445/2886] 10.222.0.62 () {58 vars in 1152 bytes} [Fri May 27 03:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:56.465351 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329219a4-64a1-4119-835c-03f5ab8f392b None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.474593 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329219a4-64a1-4119-835c-03f5ab8f392b None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.479178 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.479812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.480119 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.480394 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.481089 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.482956 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.486599 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:56.509983 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1442/2887] 10.222.0.62 () {60 vars in 1152 bytes} [Fri May 27 03:36:56 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:56.513889 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.523495 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0dbcc-e061-41db-9ec7-81df525654d5 admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.556015 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:56.580214 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.580874 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.581156 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.581432 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] PATH_INFO: `/v3/users` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.582123 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.584041 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.603936 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1446/2888] 10.222.0.62 () {60 vars in 1143 bytes} [Fri May 27 03:36:56 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:56.607977 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.618514 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537f2fe-52e7-4200-89f2-f7fafe987985 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.623260 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.623955 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.624240 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.624515 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.625244 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.627513 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.637303 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1443/2889] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:56.640873 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.650362 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1984e86-289c-4ba4-a01b-900c75d3a9cd admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.654847 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b25a1e-34e5-4e0b-965c-0c538b2d5310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.655482 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b25a1e-34e5-4e0b-965c-0c538b2d5310 admin admin] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.655766 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b25a1e-34e5-4e0b-965c-0c538b2d5310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.656090 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b25a1e-34e5-4e0b-965c-0c538b2d5310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.656811 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b25a1e-34e5-4e0b-965c-0c538b2d5310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.658886 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b25a1e-34e5-4e0b-965c-0c538b2d5310 admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.669202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1447/2890] 10.222.0.62 () {58 vars in 1121 bytes} [Fri May 27 03:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:56.672904 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.683664 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28726fb-8c35-4143-b90d-76ebed940fb1 admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.688305 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.689172 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] REQUEST_METHOD: `PUT` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.689453 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.689738 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] PATH_INFO: `/v3/projects/6c9b9c4461c04a3b822b26172d075ee3/users/1926f30a947a4950880e99e740943ebc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.704118 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9b9c4461c04a3b822b26172d075ee3, user_id=1926f30a947a4950880e99e740943ebc, role_id=0ca4c2474a5147bbb6bcd63248ea8ef8)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.706816 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.719377 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1444/2891] 10.222.0.62 () {60 vars in 1482 bytes} [Fri May 27 03:36:56 2022] PUT /identity/v3/projects/6c9b9c4461c04a3b822b26172d075ee3/users/1926f30a947a4950880e99e740943ebc/roles/0ca4c2474a5147bbb6bcd63248ea8ef8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:56.730817 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcf815-f177-4993-88d3-d802209bae72 None None] REQUEST_METHOD: `POST` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.731114 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcf815-f177-4993-88d3-d802209bae72 None None] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.731399 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcf815-f177-4993-88d3-d802209bae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.757282 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.auth.core [None req-4dfcf815-f177-4993-88d3-d802209bae72 None None] MFA Rules not processed for user `1926f30a947a4950880e99e740943ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80273) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:56.758474 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-4dfcf815-f177-4993-88d3-d802209bae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:56.826705 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1448/2892] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:36:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:56.837273 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:56.838962 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:56.897714 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13fa9d-4785-456d-a79b-f99486b6380b admin admin] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:56.901299 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937121bf-c4b0-4eb1-a12a-96e444076196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a8b4111f8774b03815a525fdbdaa4e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:56.901934 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937121bf-c4b0-4eb1-a12a-96e444076196 service cinder] REQUEST_METHOD: `GET` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:56.902217 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937121bf-c4b0-4eb1-a12a-96e444076196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:56.902494 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937121bf-c4b0-4eb1-a12a-96e444076196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:56.911254 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937121bf-c4b0-4eb1-a12a-96e444076196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:56.912698 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937121bf-c4b0-4eb1-a12a-96e444076196 service cinder] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:56.921601 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1445/2893] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:36:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:06.823740 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:06.906848 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcf815-f177-4993-88d3-d802209bae72 None None] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.917869 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:06.971996 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcf815-f177-4993-88d3-d802209bae72 None None] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.976656 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.977521 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.977804 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.978077 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] PATH_INFO: `/v3/users/1926f30a947a4950880e99e740943ebc` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.979243 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1926f30a947a4950880e99e740943ebc)` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.981162 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.026407 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1926f30a947a4950880e99e740943ebc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.027043 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1926f30a947a4950880e99e740943ebc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.034467 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1926f30a947a4950880e99e740943ebc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.035074 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1926f30a947a4950880e99e740943ebc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.046780 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.notifications [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1926f30a947a4950880e99e740943ebc'} {{(pid=80273) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.059148 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1449/2894] 10.222.0.62 () {58 vars in 1223 bytes} [Fri May 27 03:37:06 2022] DELETE /identity/v3/users/1926f30a947a4950880e99e740943ebc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:07.063586 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] REQUEST_METHOD: `POST` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.063900 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.064184 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.081542 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.auth.core [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] MFA Rules not processed for user `582245e4f2eb472dbbc5854ee9bc3452`. Rule list: `[]` (Enabled: `True`). {{(pid=80272) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:07.082658 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.155264 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1446/2895] 10.222.0.62 () {58 vars in 957 bytes} [Fri May 27 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:07.165483 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.166857 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.223733 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6bc6a-82fc-4dbb-bd1d-d4295e31421d admin admin] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.227419 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a364b6179abb4ad0883898090acef827', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.228105 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.228415 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.228692 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.238697 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.240202 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.249558 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1450/2896] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:37:07.507812 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] Authenticating user token {{(pid=80272) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.509241 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.fernet_utils [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80272) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.522215 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe352-c897-4be9-8039-6a101d718113 None None] Validating token access rules against request {{(pid=80272) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.527249 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '582245e4f2eb472dbbc5854ee9bc3452', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd52b88b861a746ccb689f0f6a32f3ad1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80272) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.528156 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.528436 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.528711 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] PATH_INFO: `/v3/projects/6c9b9c4461c04a3b822b26172d075ee3` {{(pid=80272) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.529902 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9b9c4461c04a3b822b26172d075ee3)` {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.531839 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] RBAC: Authorization granted {{(pid=80272) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.577481 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9b9c4461c04a3b822b26172d075ee3', 'request_id': 'req-46dac27b-ff4b-4334-a655-f38c791392c4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.584543 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9b9c4461c04a3b822b26172d075ee3', 'request_id': 'req-46dac27b-ff4b-4334-a655-f38c791392c4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.590147 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9b9c4461c04a3b822b26172d075ee3', 'request_id': 'req-46dac27b-ff4b-4334-a655-f38c791392c4'} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.602144 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] The token cache is being invalidate because project 6c9b9c4461c04a3b822b26172d075ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80272) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:07.602433 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.603063 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG keystone.notifications [None req-46dac27b-ff4b-4334-a655-f38c791392c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80272) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.604626 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: [pid: 80272|app: 0|req: 1447/2897] 10.222.0.62 () {58 vars in 1232 bytes} [Fri May 27 03:37:07 2022] DELETE /identity/v3/projects/6c9b9c4461c04a3b822b26172d075ee3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:17.245433 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:17.599141 dv-cinder-828807-4-iscsi devstack@keystone.service[80272]: DEBUG dbcounter [-] [80272] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80272) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:10.382760 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] Authenticating user token {{(pid=80273) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:10.384860 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:10.472274 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258db9c1-ba98-4741-b6b3-df2a284b8895 service neutron] Validating token access rules against request {{(pid=80273) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:10.539031 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e77d45-0fcb-4fb0-897b-640b604c19c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e6a5d72a7da4b07ac309bf308fc4745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '246f0e2eb19d40088d49d107007110e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80273) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:10.540245 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77d45-0fcb-4fb0-897b-640b604c19c4 service placement] REQUEST_METHOD: `GET` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:10.540717 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77d45-0fcb-4fb0-897b-640b604c19c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:10.541191 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77d45-0fcb-4fb0-897b-640b604c19c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80273) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:10.544010 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.fernet_utils [None req-a3e77d45-0fcb-4fb0-897b-640b604c19c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80273) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:10.556370 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e77d45-0fcb-4fb0-897b-640b604c19c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:10.557941 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e77d45-0fcb-4fb0-897b-640b604c19c4 service placement] RBAC: Authorization granted {{(pid=80273) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:10.567556 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: [pid: 80273|app: 0|req: 1451/2898] 10.222.0.62 () {60 vars in 1365 bytes} [Fri May 27 03:38:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:38:20.563381 dv-cinder-828807-4-iscsi devstack@keystone.service[80273]: DEBUG dbcounter [-] [80273] Writing DB stats keystone:SELECT=67 {{(pid=80273) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}